A|B Testing Guide and Best Practices

By Tinuiti Team

Jul 03 2014

One growth opportunity that often gets overlooked is right at our disposal and able to be implemented without any other resources – at no extra cost! When thinking about growth, instead of thinking about areas we can spend more by adding more keywords or partnering with other vendors, we should take a few steps back to Search 101 basics and ensure that we are fully confident that the ad copy we have in our account right now is the best place it can be. Of course, it’s always changing and ad copy testing definitely should always be an ongoing initiative in your account as a best practice. I can promise you, by engaging in ongoing ad copy testing you’ll definitely see stronger CTRs and more volume to your account. A|B testing is the most effective way to test the ads in your account and come to a strong conclusion what works best.

What is A|B Testing?

The main goal of A|B testing is to identify and discover the top performing ad copy based on your goals. A|B testing is randomized experiments that occur in a controlled setting. During an A|B test, only one variable changes at a time with copy rotated equally, thus receiving the same number of impressions.

Getting Started on A|B Testing

Goal Setting

The first step in starting with A|B testing is clearly understanding your goal. Your goals for the A|B test could be different than what the main KPI for the account is. For example, my retail client always has ROI as their main KPI and goal, yet for ad copy testing purposes I typically look at CTR as the goal. (I want to ensure I have the most enticing ad that gets clicked on the most to bring more traffic to the site).

Some goals for A|B testing could include: Margin, conversion rates, CTR, ROAS/ROI. With no surprise, the area that gets the most amount of volume will give you the fastest results and will be faster to hit a strong confidence level for the test. For example, a goal for CTR will drive faster results than a goal for ROAS.

Know What You Are Testing

It’s important to have clear objectives and understand what exactly you will be testing. That said, it is important to have a “control ad” – the ad that you will have as a starting point to test the other ads against it. Typically my control ad is one that I’ve seem perform the best in the past and my variables are purely variations against it.

What to test? The possibilities are endless with A|B testing. Get creative with your tests, you can test out each and every single aspect of your ad – headline, description line 1 and 2, display URL. A simple change can make a difference! No test is too small, maybe even consider testing out something as small as punctuation – adding a period vs. not.

The most common testing areas will include:

A different emphasis and messaging:
- Product description
- call-to-action
- promotions/offers
Including price vs. not
Including brand name vs. not
Variations of words or brand name (example: including the registered trade mark with the brand name)
Including variations of keywords in ad (in headline or display URL)

A great place to get started is looking at historical ad copy and making some analysis of what’s worked or hasn’t worked well in the past.

Best Practices of A|B Testing

Test only one variable at a time. For you to clearly understand what is working you can and should only be testing one at a time
Always remember to keep search ad copy and creation best practices in mind. It’s great to get creative but don’t lose sight of what works – make sure that you still maintain relevancy in your ads to the keywords.
Start small – you don’t need to run a test across your entire campaign. In fact, I recommend not doing that. It’s not only hard to keep track of, but if something isn’t working in an ad variation you don’t want it pulling down the overall performance of the entire account. Choose a handful of your highest traffic driving ad groups and start there. By choosing top traffic drivers, this will help you get to the results as quick as possible. Once you find improving results, then you can roll that strong ad out cross all ad groups.
Recommended to test between 2-6 variations of ads at a time. Of course, the more variations you have rotating the longer it will take to achieve results that are statistically significant. If you are testing on ad groups that have lower volume, it’s recommended to not test more than 3 ads.
How long do I test for? Not a question I can answer for you. Testing length is completely dependent on the point above – how many variations you are rotating out and how much volume you have
Make sure you have changed the settings in the engine to get the ads to rotate equally. You will need to ensure you get an equal number of impressions
Once you determine a winner you are not done – there is always more you can be doing. Now it’s time to move on to another test!
Keep a log – with so many tests you’ll need a clear and organized way to remember what’s worked well in the past
If using a bid management or 3^rd party tool, utilize dimensions/tags/labels to help you keep track of the ads that you’re testing. These will help you organize and quickly identify the varying ads that can sometimes look very similar. It’s much easier to pull performance this way!

Reporting and Analysis

It’s great that you have a strong understanding of what goes into an A|B test but what’s equally important is analyzing the data and reporting what’s worked. Your test is an absolute waste if you can’t compile good insights or information on it and share it with your client.

To come to a conclusion about your test results you’ll need to make sure you’re confident in the data you have (in other words, you need to make sure you have enough data to make a decision). Using a statistical calculator is the best approach to take here. It’s recommended that you at least hit 90% in confidence to make a decision about your ad. An Elite favorite resource is: http ://www.cardinalpath.com/resources/tools/ppc-ad-testing-tool/

Golden Rules to Keep in Mind While Testing

Be patient, especially if your campaign has low volume, it might take time to come to a strong confidence level and see results
Only test one variable at a time

Always be testing! There is never an end to the possibilities of tests