Learn how to run an A/B split test: identify what you want to test, choose the sample size, determine the winning version, and establish the test duration.
Alternatively, split the audience into two equal parts and send a variant of the communication.
How to activate the A/B split test feature
To start an A/B split test, you need to activate this specific option.
Go to the "Journey Lab" menu and access the tab to edit the draft communication.
Click the Setup button and enable the "Conduct an A/B split test" option.
You will need to choose between two A/B split options:
- 50/50 (default choice): divide the audience into two equal parts and send a variant to each of the two parts
- Test and send: send A and B variants to a percentage of the audience chosen by you, identify the best version and complete sending the winning version to the remaining part of the audience
A/B split - Test and send
With the Test and send option you can choose what to test among: the subject (default choice), the sender, or the content of the communication.
If you opt for testing the subject, you will need to create two different variants of:
- Subject
- Preview text
You can use similar or completely different words in the two variants, at your discretion.
If you decide to test the sender instead, you must define two different versions of:
- Sender name
- Sender email
Here too, you have complete freedom in choosing the two senders to test.
Finally, if you want to test the content, prepare two different versions of the message.
In the communication design, select the version from the dropdown menu at the top and define the content
or, if the two versions differ only in small details, work on a single version (e.g., A), then switch to the other (e.g., B) and use the "Copy from version..." function
This way, you can save time by modifying only certain elements without creating a second version from scratch
The communication card displays the summary information for each version
allows you to preview each of them
calculate its weight
and the spam score
Choose the sample size
Decide the percentage of contacts that will be part of the sample for the two test versions (10% is a common choice).
The sample is randomly selected from your audience and is divided into two equal parts (A and B), to which you will send the two versions of the communication.
For example, if your sample is 10%, 5% will receive version A and another 5% will receive version B.
The winning version
Decide how to determine which of the two versions (A or B) is better at the end of the test.
- If you are testing the subject line or sender, "Open rate" is a good metric to consider. For content, "Click rate" is preferable.
- With "Manual choice," you will receive a notification at the end of the test with the results, and you can decide which version to send.
Duration of the test and sending the winning version
Choose how many hours you want the test to last (usually it's 24 hours), after which:
- If you've chosen "Open rate" or "Click rate," the winning version will be sent to the rest of your audience.
- If you've opted for "Manual selection," you will receive a report via email with results to help you decide which version to send to the remaining audience.
For "Manual selection," click the "Select and send" button in the email received.
Access the communication report and evaluate the results obtained with the A/B split test.
Choose the winning version by clicking "Send version ..."
A window opens where you can set the day and time you want the winning version to be sent to the remaining contacts, i.e., those not part of the A/B split test sample.
Click "Send" to confirm the delivery.
Alternatively, if you decide to complete the send before the end of the test, access the "Deliveries" list under the "Monitor" area and select "Complete sending" from the available options by clicking the three dots on the right.
You will then access the A/B split communication report shown earlier, where you can follow the steps described above to send the winning version to the remaining contacts.
A/B split - 50/50
Even with the 50/50 option, you can choose what to test between: the subject line (default choice), the sender, or the content of the communication.
However, in this scenario, you do not have the option to select the sample or determine the duration of the test. Additionally, for logical consistency, a winning version is not identified, as the entire target audience is divided into two equal parts for the communication delivery.