How to A/B test flow branches
You will learn
Learn how you can A/B test flow branches using a multi-branch split to test things like timing, discounts, and number of emails within a flow. In this article, we run through how to set up and end an A/B test for a flow branch.
You can also A/B test a single flow email. Read how to A/B test flow emails for guidance.
Set up a test branch
- Navigate to the flow you would like to A/B test in the Flows tab.
- Add a split where you would like to start your A/B test.
- When configuring the split, select Random sample as the condition. Here, you'll set what percentage of your audience goes down the first path, which you can think of as the control. The percentage applies to the profiles evaluated for that path.
- For an even A/B test, select 50%. Otherwise, click the dropdown to choose a different percentage.
The split will randomly choose a branch per profile. Because this is completely random, you may not see a completely even amount of profiles in each branch with a 50% split, but it will be close.
For example, you may want to test sending an additional email to one path of your welcome series. To do this, add a split at the end of your flow, add a time delay, and then configure the email content.
Once you have your split configured, you can build out the content of your test path. Depending on what you're testing, this could be several emails or different timing from what you have in your control path. You can also test more than two variants at once by adding more paths to the split — a control plus several test paths.
When running an A/B test, bear in mind that testing more than one variable at a time can skew results and make it difficult to determine how to attribute any differences in revenue, open rates, etc. For this reason, it's a best practice to test variables one at a time after you've determined a winner.
Apple Mail Privacy Protection (MPP), which was released with iOS15 and updates to other Apple devices, may lead to inflated open rates due to changes in how we receive open rate data.
If you are triggering flows off of opens themselves, we suggest creating a custom report that includes an MPP property to review these affected opens. You can also identify these opens in your individual subscriber segments.
Determine the best branch
To determine which time delay is best, review the analytics for each message. Depending on the flow, decide which path is best based on the open rate, click rate, and conversion rate.
To quickly view the flow analytics for messages in your flow:
- Click the Show Analytics icon button located on the nav bar
- Review the metrics of the messages in each path and decide which path you prefer.
Learn more from our article on understanding flow analytics.
End the A/B test
After deciding which path is better for your audience:
- Click on the split.
- To have all recipients go down the control path, set the percentage to 100%; if you want everyone to skip it, set it to 0% (as shown in the example below). This way, those already in the Waiting queue still receive the old message rather than being taken out of the flow like they would if you deleted the path.
- In whichever path is no longer in use, set the messages to Draft.
Additional resources
- How to A/B test a flow email
Learn how to use Klaviyo’s A/B testing experience to test individual emails in a flow, including things like subject line, discounts, and email content, etc. You can even use A/B tests to determine if plain-text emails perform better than image-heavy HTML emails.
- Best practices for A/B testing reference
Learn best practices for using Klaviyo's A/B testing features to help optimize your communication with customers.
- Mastering Email A/B Testing: Klaviyo’s Proven Tips & Tricks to Boost Revenue