The ABCs of A/B Testing

The Importance of A/B Testing

Let’s start with a quick history lesson. A/B testing as we know it today is nothing new — it’s actually been around now for more than 12 years. Granted, it’s become much more advanced in the last decade, but the foundation and underlying principles of A/B testing remain the same. In fact, the Google engineers ran their first A/B test at the turn of the millenium to determine the optimum number of results to display on a search engine results page. It’s encouraging to note that even a technology powerhouse like Google experienced glitches during their first A/B test, which proved disastrous due to slow loading times.

But once Google perfected their strategy, they became a pioneer for A/B testing. In 2011 alone, they ran over 7,000 A/B tests. And they’re not the only company that recognizes the value of testing and tweaking variables on their sites. and eBay are also big believers in A/B tests, which is why both companies frequently present slightly altered versions of their websites to live users.

Although all of these examples focus on large companies with (almost) endless resources, A/B testing has found a place in smaller companies too. For those who may not be as familiar with the concept, A/B testing is a way to compare two versions of a single variable (“does offering a case study or a white paper result in a better click-through-rate?”). By testing a user’s response to A versus B, you can see which of the two are more effective. This allows you to adjust and tailor your site, content, emails, etc. based on performance and user preferences. Using a marketing automation system like Pardot, you can easily test variations of the following to see what kind of results you get:

    • Header images
    • Landing page headlines
    • Email subject lines
    • Email incentives and offers
    • Image content and placement
    • Landing page copy
    • Form fields
    • Form length
    • Email send time

The beauty of A/B testing is that it can be done on almost anything on a continuous basis. Users’ tastes and preferences change over time, so your marketing should be able to adapt to that. Luckily, most marketing automation softwares come with the functionality to run A/B tests on an ongoing basis using your current resources, so you can update your landing pages, forms, emails, images, and everything else to keep up with changing trends. If you want to get even more advanced, you can try running multivariate tests to compare multiple variables at the same time, much like running several A/B tests in different combinations (“Do users respond better to image A next to a short form? What about image B next to a longer form? How does this change when the landing page headline is altered?”).

If Google and are placing such an emphasis on A/B testing, it’s probably a good indication that these tests are worthwhile. Only A/B testing and continual reporting can provide such detailed insights into the intricacies of online marketing. Have you tried A/B testing and found it to be useful for your business?

Read Next

Mad Men vs. Marketers: How the Art of Influence Has Evolved [INFOGRAPHIC]

5 thoughts on “The ABCs of A/B Testing

  • Jenna, marketers have been doing A/B testing since well before the Internet existed. The one critical point you haven’t mentioned is ensuring the sample is large enough to produce statistically valid results. There are formulas for this, but a reasonable rule of thumb 30 to 50 responses for each test group.

    • That’s a great point and certainly worth mentioning. Thanks for the comment, David!

    • David — thanks for the comment. I totally agree that using too small a sample size is by far the biggest issue with B2B marketers (over)testing. Typically it is much easier in B2C because it is easier to get statistically significant results.

Comments are closed.

Ready to see how marketing automation can help you grow at record speeds?
Request a personalized demo today!
Need help?No problem. Chat with our sales experts or call us at 1-800-667-6389.