still life

Are You Running Your Facebook A/B Tests The Right Way?

By now, you’ve probably seen any number of opinions on whether or not Facebook ads work. Some say they might work for some verticals, but not for others. Others says that they do work, but with a caveat: that you must have specific goals in mind.

Of course, ads of any kind only actually work when you have a specific goal in mind, so let’s take that with the most granular amount of salt possible.

What digital marketers need to know about Facebook is that there are so many advertising options right now that just about any goal related to commerce can be realized through the proper set of ads targeted at the proper set of people.

But what is necessary for success? The same thing that is necessary for success in any other digital advertising venture: test, test, test. Split testing your ads allows you to do what every good marketer knows is necessary: to allow the audience to decide.

If you want to know what kind of ad creative, ad copy, and targeting options are going to work best, put several out there, and let your audience show you which one(s) they will engage with most.

In order to do that, it’s necessary to set up split testing, otherwise and interchangeably known as A/B testing. Let’s take a look at the value of split testing your ads, what you should be testing, and just how much testing you’ll need to know to find out the truth of which ad combination performs best.

What is A/B testing, anyway?

The concept of the A/B test in Facebook advertising is very simple: two ads go up against each other and compete to become the winner. The important thing to note is that, for a true A/B test, only one aspect of any ad must be different. You must first define the hypothesis.

Michael Aaggaard, a conversion rate optimization expert, has this to say about the hypothesis:

The main goal of A/B testing is to eliminate guesswork from your marketing optimization efforts. However, the simple act of running an A/B test is not enough to achieve that goal. If you’re testing random ideas, you’re still relying on guesswork. All you are doing is pitting two guesses against each other to see if one is better.

That means that you need to have a very clear idea of what it is that you hope to achieve from an A/B test, and this starts with the hypothesis. According to Michael, you can build a hypothesis like this:

Because we saw [data/feedback] we believe that [change] will cause [outcome]. We will measure this using [data metric].

Now you’re going to start testing one element of an ad against another ad with all else equal except for that one element. Here’s an example from AdEspresso in the image below.

facebook ad test

The one variant that they’re testing is the ad copy. The image is the same, the headline is the same, we can assume the targets are the same, but the copy is different. By testing just the copy of the ad, they’ll be able to single that one element out and find out which copy is the best for that audience.

From there, they can start to test other elements of the ads, but the important thing is that you test just one element at a time. This allows you to take the guesswork out of your testing, as might happen if you were to test multiple elements at a time.

Keep it simple. Start by defining what it is that you want to learn, and then launch that test. Once you have a good sample size, you can pick a winner, and start testing other elements.

What things should I be testing on my ads?

There are very specific elements of ads that you’ll want to test. Because you have options with Facebook that are clearly defined by its ad platform, you’ll want to test the following:

  1. Images
  2. Headlines
  3. Ad copy
  4. Call to Action

Because job number one on Facebook is getting your audience to actually look at your ads (remember, you’re competing with someone’s best friend’s new baby, a cute kitten, and the amazing lunch that guy you knew in high school just had) your images are crucial to success. They’re not the only thing, but they’re incredibly important.

As such, you might want to invest in having a few more image creatives on hand in order to really dig into what’s going to get eyes on your ads.

Once folks have seen your ads, you’re going to need to give them a reason to engage with that ad in just a few words in your headline. Spend some time on your headline and ad copy. Get creative. Take some chances. As with any other sales pitch, the worst the audience can say is “no.” If they do, your test will help you to determine how better to reach them.

All elements of your ads are important, and just as important as the creative is the audience.  Ad targeting allows you to get to the audience you want to reach based on different geographic location, gender, and a lot more. For ad targeting, you should be testing the following:

  • Country
  • Gender
  • Placement (where your ads are displayed)
  • Interests
  • Age
  • Custom Audiences
  • Relationship Status
  • Purchase Behaviors
  • Education Level

How many ads should I be testing?

Based on what you’ve just learned, you can see how this can quickly get out of hand in terms of how many ads you’re going to need in order to test different hypotheses. In the image below from AdEspresso, you can see that you could start to test many variations of your ads. You are still getting an A/B, but you’re drilling far deeper all at once, instead of having to do them one at a time.


If you’re limited in your ability to create a lot of ads to test all at once, you will have to resign yourself to testing just a few at a time. You’ll still be able to be very detailed, it will just take you a little bit longer.

When can I pick a winner?

There is a delicate balance to be achieved when testing Facebook ads. On the one hand, Facebook’s algorithm is notorious for crushing entire campaigns based on the performance of just one ad in that group. On the other hand, you’ll want to achieve some kind of statistical significance before signing off on one or another ad.

How do you know you’ve achieved statistical significance? This can be difficult, especially for advertisers with smaller budgets. It’s really a matter of keeping a close eye on performance and getting as close as you can to a number you can generate from a statistical significance calculator. I like this one.

Just enter the values in the calculator and you’ll find whether or not you can be confident based on the number of clicks and conversions you’ve gotten. It can be a time-consuming process, but well worth putting in the effort – unless you’ve got some kind of tool that can do this stuff for you.

When should I start testing my Facebook ads?

The answer is NOW! Any digital ad campaign of any kind, be it on Facebook or elsewhere, that is not actively testing is doomed to failure. The more you know, the better your advertising efforts will be.

The more ads you create, the better your information will be. The more highly defined your tests are, the better you’ll be able to define what works best and what resonates with your audience. Each test leads to another, which allows you to keep getting better results all the time.

Never stop testing, and never stop striving to reach your perfect audience.


If you’re ready to take your Facebook advertising to the next level, make sure you download a copy of our advanced Facebook advertising techniques webinar and eBookcomplete with the Facebook Marketer’s Checklist!

Mark John Hiemstra

Mark John Hiemstra

Mark John Hiemstra is a content and social media marketing consultant to Acquisio. He got involved in digital marketing purely by accident, and has spent the last several years learning about building relationships with consumers online through the many channels available to those who are willing to get involved. He is loathe to discuss himself in the third person, but can be persuaded to do so, from time to time.

The First Machine Learning Marketing Platform
Built to Scale Search for Local Resellers & Agencies

Automate, optimize and track more campaigns, more profitably.