How to AB Test Ideas with Meta Ads

Meta is an amazing tool for real time customer feedback on ideas, but most people don’t leverage the platform for this purpose.

In this article we’ll cover the following:

  1. How to structure your Meta campaign to test discrete concepts
  2. Pitfalls to avoid
  3. Statistical methods to make sense of your data

Meta is an amazing tool for real time customer feedback on ideas, but most people don’t leverage the platform for this purpose. Out of the box, Meta allows AB testing of individual creative or ad settings. This doesn’t work for testing high level ideas/concepts.

Here’s where most people go wrong:

You want to test idea A vs idea B. So you put together 1 creative asset for each. The winner is the better idea right? Maybe…

Really what you’re comparing is ad creatives and copy against one another. These results will be only loosely correlated to the actual capability of the idea.

This campaign structure doesn't take advantage of ad set optimization.

How we solve this:

Leverage ad set optimization and several creatives.

Ad set optimization means giving Meta multiple ads and allowing it to allocate budget to optimize for conversions. Basically, giving yourself multiple shots on goal.

Produce multiple creatives for concept A, multiple for concept B and put them in their respective ad sets.

Now, when we compare results we remove some of the noise associated with individual creatives allowing us to compare concepts rather than creatives.

This campaign structure takes advantage of ad set optimization.

How to read the results (statistical methods)

You don’t want to just take the bigger number and say that’s the winner. We want to make sure our results are statistically significant.

Our default method is a z-test on conversion rate.

The “Conversion” you select is a design decision. It can range from a “click” (easiest, but least accurate) to an actual purchase (hardest, but most accurate).

If you just want the answer, use this! Validated AB test, Stat Sig Tool

If you found this interesting

If you made it here you probably care about growing your business and learning about what users want.

Using Validated accomplishes both. We provide AI enabled marketing services to acquire you customers via digital ads. The methods above are a tiny subset of the tools we use.

Book a demo here: https://www.getvalidated.ai/

VALIDATED AI ICON

Learn what Customers Want.