You don’t need A/B testing

I dare you to come up with a more spiteful question to type into Google Meets, as you take the last bite out of your donut, with the mic and camera decidedly turned off.

The question is followed by awkward silence until someone steps in and grudgingly says, “Sure, we can do that.” And then they make a note of one more thing to do amidst their pile of work because, well, it sounds like the smart, objective thing to do.

Truth is most things don’t need to be A/B tested. That’s because most things don’t have the scale required for a meaningful impact on conversions.

There are exceptions.

Say you are an FMCG brand running a multi-million ad campaign and need to optimize the CTA—by all means, test the colour of the “I’m in” button.

Or if you have a B2C product with millions of users, then yes, split testing components will have a meaningful impact on your results.

Even so, there’s plenty of research that you can lean on instead of reinventing the wheel. For example, a red CTA button is shown to drive 21% more conversions than a green one in totally identical conditions. So maybe you don’t actually need to repeat the exercise in many cases and a few Google searches will do the trick. But I digress.

It’s when B2B companies that get 10,000 page views a month to their website try to A/B things when you enter murky territory.

In my experience, this is usually suggested for one of the following reasons:

1. You don’t understand the business and the unit economics well enough to not make such a proposition

2. You want to sound smart and feel like you’re making a meaningful contribution

3. You personally don’t like the font size, colour, or placement that is presented as the default and assume that most other users have the same sensibilities as you

4. Or the worst reason of all: You don’t like the person leading the project and just want to create roadblocks in their way

Whatever the reason may be, you need to understand the business context and the available scale before proposing to A/B test things.

A/B testing shouldn’t be used as a crutch to settle arguments. In the vast majority of cases, it doesn’t actually matter if you pick Serif or Sans-serif, or the colour blue or orange, or the size 14px or 16px.

Marketing leaders should rely on their intuition and good sense to make these calls based on the medium and the established brand identity.

Of course, A/B testing companies have been around for a long time now. They have spent a lot of time finding all the different users they can potentially sell their solution. I bet many have a user persona template for Brian the B2B marketer.

So, before you suggest A/B testing anything, think about how useful it’s likely to be, given the unique situation you are in and the associated scale.