• home
• resources
• whitepapers
• maximize email campaign success

About Split Testing

Split testing is a simple, yet extremely important process. It should be a standard part of your email marketing procedure and it should be performed before each campaign deployment. Split testing allows you to test different aspects of your message with random test groups before you deploy the message to your entire list. That way, you'll have proof of what works best so you can send that version to the rest of your subscribers. A/B split testing is a test between two different versions of your message, whereas multi-level split testing allows you to test three or more versions.

For example, if you wanted to send all of your subscribers an email offering free shipping on purchases made that day, you can use split testing to test different subject lines to see which one generates the most responses. Because you are only testing one small area of your email, setting up the test emails is quick and easy.

The first step is to set up the test segments. This takes less than one minute using Listrak's profiling tools. You simply define the percentage of your subscribers that should receive the test versions and the number of test versions you'd like to send. The system then selects a random sample of email addresses and randomly splits them into the specified number of test groups. If you have 153,698 subscribers and want to test four different subject lines using 10 percent of your list, Listrak will generate four test groups of 3,842 contacts. If you're not sure what percentage of your list you should test, Creative Research Systems offers a useful online sample size calculator.

You can get more specific by setting up test groups based on a segment of your list. Segmenting your lists is an email marketing best practice as it greatly increases the relevancy of your messages and overall ROI and Listrak gives you an easy way to test each segment of your list by simply defining the segment you wish to target and then setting up the test groups. For example, if your list of 153,698 contains 73,572 females between the ages of 25-40, and you want to test four different subject lines with 10 percent of that segment, each group will have 1,839 contacts. Again, because Listrak is randomly generating the test groups, this process takes less than a minute to set up.

Creating your test messages is just as quick. You simply create the message and select the test group to receive that message. Using the above example, you could generate four versions of the same message with different subject lines, such as "Free shipping on all your favorite products - today only!", "Shop today and receive free shipping on your entire purchase", "Shop now and the shipping is on us!", and "Today only - extra discounts + free shipping". The time investment to create the different versions is kept to a minimum as you only have to create the message once. Just change the subject line, select another test group, resave the message, and deploy the new copy to the subscribers in the test group.

The fast response that email generates makes split testing so effective. Over 74 percent of all responses are received within the first 24 hours, with nearly half of those coming in within the first several hours of the deployment. This means you can get accurate results of the tests within one day. The metrics that you measure should depend on the goal of your email, so be sure to monitor the correct results. If you're testing subject lines you're probably testing to see which version of the message generated the most opens; however, if you're testing calls-to-action you're probably testing to see which version generated the most click-throughs. While it's important to look at the overall numbers of each email, you must be sure to focus on the metric that measures what you set out to achieve. As you can see from Listrak's Message Activity Report, the version with the subject line "Today only - extra discounts + free shipping" outperformed the other versions, so that would be the version that is deployed to the remaining subscribers in the segment.

Listrak's account management and professional services teams can also assist customers with this process. Our experts can perform and evaluate the tests on your behalf. If you have questions, or if you would like to see a live or video demonstration of how split testing works within Listrak, contact one of our email marketing experts today.