A/B testing allows you to optimize message performance by testing different versions of triggered messages. You can test up to six variants within a single A/B test to compare elements such as message copy, MMS vs. SMS, and wait times.
Add an A/B Test to a Journey
- Drag the A/B test node onto the journey path from the left-hand menu or select it from the pop-up menu.
- Select Split traffic evenly or Customize traffic split.
- Click +Add variant to add up to six variants. Ensure the total split equals 100%.
[Beta] Define your winning criteria and choose how the test should end:
- Automatically: The test ends once a metric reaches statistical significance based on your chosen criteria.
- Specific duration: The winner is determined based on the best-performing variant at the end of a set time period, regardless of statistical significance.
- Manually: You determine when to end the test.
- Click Save.
Notes:
- To ensure sufficient data for accurate results, it is recommended to run A/B tests for at least one month.
- Nested A/B tests allow you to run a secondary experiment within an active test group. Please be aware that because this further segments your audience, it will take significantly longer to reach statistical significance than a standard test.
Track Performance
To view the results of an active A/B test, click the Send text message step within a variant to open the Message performance pop-up.
[Beta] Determine and Promote a Winner
Once a test concludes, a banner appears in the journeys list view. Click the link in the banner to navigate to the journey, where the completed test is labeled for identification.
Click the A/B test node to view variant performance and the identified winner. From here, you can select Promote the winner.
The A/B test that has ended will also be labeled within the journey for easy identification.
What Happens When You Promote a Winner?
Selecting Promote the winner creates a new journey revision in a draft state. In this revision:
- The A/B test node and all losing variants are automatically removed.
- The winning variant remains as the primary path.
- Once you publish the new journey revision, all subsequent subscribers entering that branch will receive the winning variant.
Manual Alternatives
If you prefer not to use the automated promotion tool, you can manually update your journey using one of these methods:
- Adjust the split: Edit the journey and assign the winning variant 100% of the traffic.
- Reconstruct the path: Drag the winning message directly onto the journey path and delete the A/B test step.
Both manual methods will create a new version of the journey, which you can track via the version dropdown menu.