Tel: 01865 242425

Tel: 1-855-835-0471

Tel: 1300 993 579

Getting started with email testing and subject lines

Ryan Phelan

We recently published the 2015 Subject Line Analysis Report and I’m excited to share some of those results in my inaugural blog post, as well as some insights on testing email campaigns beyond just the subject line. I’ve been in the email marketing industry for 15 years, and I look forward to becoming part of the Adestra community, offering tips and tactics from my email experience.

Insight you don’t want to miss

The data behind the 2015 Subject Line Analysis Report was generated with our anonymized client data including:

  • more than 125,000 campaigns
  • more than three billion attempted email sends
  • more than 400,000 email opens
  • four different industries

Why should you care? Subject lines are a uniquely testable aspect of email marketing because they appear in the recipients’ inbox regardless of whether the email is opened or not.

The top performing keyword across all industries? “Thank you” at 62%. This phrase tends to be used in automatically triggered emails which typically get a higher open rate because they follow an action taken by the recipient, and they get the timing right.

Whatever you test, cover these two bases

The Subject Line Report gives you great ideas for testing your subject lines, but keep in mind that the key metric they impact is open rates. If you want to improve open rates and engagement, you need to experiment with other elements of the email, such as copy or design.

When testing any aspect of your digital marketing it’s imperative to strive for statistical confidence in the results. This covers two bases:

  • making sure your testing sample is large enough to provide valid results
  • checking if the change in results from the control version to the test version is large enough to be significant

Without these two conditions being met, you’ll end up with more of an educated guess, than a statistical decision. The good news is that you don’t need to be a statistical whizz to get these right, there are plenty of calculators online that can help!

Getting testing ideas

Now that you’re in the mood to experiment, you need some inspiration, right? There is a wide range of industry media out there sharing ideas and providing testing advice, but make sure you are learning from a reputable resource – not all are great. Two that I like (among many) include MarketingSherpa and Marketing Profs. Both have proven over a number of years to be solid resources for marketing information, including testing.

Reach out to your trusted peers and find out what industry resources they regularly read and turn to. The main thing is to understand there are both good and bad voices out there. Define what you believe about email marketing and align yourself with those resources that resonate your beliefs.

Be sure to watch for part two of this post that will feature some testing tactics to help you improve your email marketing. Coming soon!


Tags: , , , ,

  • Ben Rothfeld

    Good stuff, Ryan. Do you have any suggestions on getting the right sample size? I used to use 5,000 per cell as a default, but that might be a bit big for some readers.

  • Ryan Phelan

    Hey Ben. In the next blog, we actually go into sample size, but as a teaser, there’s no perfect answer to the sample size question. To get a reliable confidence interval, it’s 20,000 at a minimum and 40,000 to get close enough to actually make a good call. The lesser you get, the less “true” the result is because the statistical difference is greater than comparing 2 numbers. it’s a bit big, but if you don’t get that large of a audience, then you’re not going to get statistically relevant.