Use A/B test campaigns to test up to 4 variations of the same message. This will help you determine which messaging converts the most subscribers into paying customers. There are 2 types of A/B tests:
- Message content: Test up to 4 variations to see which one has the highest click-through rates or lowest unsubscribe rates.
- Message send time: Test up to 4 send times to see which time of day has the highest engagement.
If we can’t determine which of your message variants or send time variants had the best performance, we’ll select one of your messages at random and use it as the winner.
You’ll get the best results if you run A/B tests with large numbers of subscribers. As a general rule, more subscribers leads to better test data.
Create an A/B test campaign
Watch the video, or follow the step-by-step instructions below to create an A/B test campaign.
- At the top-right corner of the Campaigns page, click + Create Campaign.
- Select A/B test, then Continue. The Set up your A/B test page appears.
- In the A/B test name field, enter an internal-facing name for the test.
- Below Who will you send this A/B test to? select a segment of subscribers to send this campaign to.
- For best results, we recommend sending each message variation to at least 500 subscribers.
- (Optional) Add tags to your campaign to help track the subscribers.
- Select Use Smart Sending to exclude customers who recently received a one-time message.
- Click Save & continue. The platform will direct you to the Test Details tab.
After you have created the initial campaign, follow the steps below to test the content or send times of your message.
Test the content of your message
To assess the content of your messages, you can configure unique copy and media for each variation. Then, you can divide the test results by one of the following:
- Even split A/B test: Evenly divide message variations among recipients.
- Automatic winner A/B test: Run an initial test on one portion of recipients to see which variation performs better. Then, send the winning variation to the remaining recipients.
For each test, you configure which criteria determine the success of your variation. The remainder of your segment receives only the winning variation.
Follow the steps below to configure a content test:
- After you create a campaign, navigate to the Set up your test page.
- Below How many content variants would you like to test?, select the number of message variations you would like to configure.
- For optimal results, each variation should contain 500 recipients.
- Below What percentage of your recipients should receive the test?, choose what percentage of subscribers will receive the test. You can drag the slider left or right to adjust the percentage.
- We recommend setting it to 20%.
- If you set the percentage to 100%, the subscriber lists will be divided into equal segments for each variation. For example, if you have 2 variations, the lists will be divided into segments of 50% and 50%.
- Select the winning criteria for the test.
- Below How long should we run the test before sending the winning variation?, select the number of hours you would like the test to run before determining the winning text.
- Click Save & continue.
After you configure your A/B test campaign, you can design the messaging of each variant.
Test message send times
You can configure your A/B test to assess which times of the day, or which days, have the best conversion rates. However, you can only test SMS messages; to test MMS messages, we recommend testing for message content instead.
Follow the steps below to configure an A/B test for send times:
- When you create a campaign, select Send time below What would you like to test?. When you finish creating the campaign, click Save & continue to go to the Set up your test page.
- Below How many send times would you like to test?, select the number of times you would like to send this to different subscriber segments.
- For optimal results, each send time should contain 500 recipients.
- Below Send Times, select whether you would like to send messages in your time zone or the subscribers’ time zones.
- For each variant, configure the time and date you would like to send it.
- Click Save & continue.
After you configure your A/B test campaign, you can design the messaging of each variant.