Email testing: how to plan and organize your program
In my first blog post I highlighted parts of our 2015 Subject Line Analysis Report and covered some basics on email testing, including the importance of statistically valid tests, why you should be testing more than just the subject line, and the importance of reputable advice when looking for inspiration.
This post will go into more detail, covering some tactics and insights I’ve picked up over my 15 years in the email marketing world, to help you improve your email testing.
Organize your testing program
One problem I see too often is marketers not keeping track of the tests they run. Make a binder, create a spreadsheet, whatever works for you. But you need to track at least these five basic elements:
- what you tested
- date of the test – your audience changes, so tests should be re-run
- why you tested that element – what is your hypothesis?
- the results of the test
- the action taken as result of the test
To get more specific, you could also include a screenshot of the email tested and links to the asset library to keep a visual record of the creative elements.
Connect the dots in your email testing strategy
If you’re going to take the time to conduct testing, make it worth every minute by:
- making sure you’re running statistically valid tests
- not testing more than one element of an email at once (unless you run multivariate tests, which we cover in our free guide)
- not running a series of one-off tests. Otherwise, you’re only proving that it worked for this email on this day
- making a three-month rolling plan on what to test based on the results of previous tests. This will help organize the process rather than finding your team testing random elements of different email campaigns.
You should keep your test tracking internal to the testing team, but at the end of every quarter put together a report. Present it to management and recap what the testing has accomplished, what was learned and what was improved.
Understand the limits of your testing audience
The subject line is an easy email element to help you get started with testing, but it mostly impacts your open rate. If you want to also improve click through you’ll have to test other email elements such as the headline, the call-to-action button or the length of copy. Our paper ‘Optimize your Email Campaign Performance with Testing’ covers what you can test, and much more in a quick guide that you can keep at hand. It even comes with a handy planner at the end too!
Just keep in mind that list size will impact what you can achieve with your tests. As I’ve previously mentioned, you need to make sure your database is statistically large enough to support your test, and then the change is statistically significant to eliminate rash conclusions.
Marketers need to be honest with themselves when planning the complexity of tests, and be willing to get to basics right even if that means focusing on list building for a while.
If you’re just getting started with email testing, ask for help from your ESP. Your Account Manager should be more than happy to schedule a call to give you strategic advice and make your testing program successful.