Marketing Experimentation Culture: 5 Easy Steps to Embrace A/B Testing

The Benefits of Building a Marketing Experimentation Culture

A year ago HBR dedicated its cover to the idea of building an experimentation practice in corporations (the article is available here).

The topic is very close to my heart. I’ve championed experiments at Mars for almost 10 years, since my first days as part of the global marketing insights team.

Marketing experimentation culture first appeared in companies from the tech sector but was embraced widely elsewhere in subsequent years. Why? Because it primarily replaces gut-feel decisions, with robust data-informed ones. All this was possible following an explosion of agile experimentation methods and big data availability.

You can’t build a solid experimentation practice without a robust and fast measurement method. A/B testing became the poster child of any sales or product-focused team. Pioneered by tech giants like Google, Netflix, or Booking.com, it is considered the gold standard indication of causation between stimuli and effects. But before tech came into play, A/B was born in the field of drug trials, where precision and quality of decisions is sometimes a choice between life and death.

How can a consumer brand build a marketing experimentation culture?

Step 1 – Embrace evidence-based marketing

You can’t create a culture of experimentation if you don’t believe in facts. Trust numbers, before known orthodoxies. You will be occasionally puzzled but will feel liberated at the same time.

A recent poll I ran on LinkedIn, told me that in today’s data right world, my network leans more towards making more gut-feel decisions instead of capturing more data. It is true that we are overwhelmed by facts, and more or faster is rarely the answer to our quest.

The right way to build a marketing experimentation practice is to select the best in-class information on your customer, and not look back.

Step 2 – Select best-in-class testing methods

A/B testing is what you should be looking at. If the pharmaceutical industry is running on A/B, why should you look elsewhere?

Programmatic media, loyalty cards, and the growth of e-commerce will enable more opportunities for easier A/B testing. For CPG companies, like Mars, this is massive, since the final sales data is never owned by the brand owner. On the flip side, for any direct-to-consumer company, where knowing the customer is at an intimate level, A/B is already a common practice.

But what can you do when A/B testing is not possible, when the audience can’t be pre-selected, and targeting is not possible? Attempting to create a natural experiment post-factum will get you as close as possible to the A/B mindset. Don’t worry; it will be messy and noisy, but the strength of the method will prevail and enable better decisions.

Step 3 – Democratize A/B testing

A/B is not controversial, relatively simple to understand, and easy to embrace by markets.

Yes, it comes with some disadvantages like a very narrow short-term outlook of impact, or a need for experimental design when you want to test multiple confounding variables, it is still the most accurate method of getting to the truth.

Spread the message, evangelize the use case, find the partners that can help you outside and let your teams run wild, within the framework.

Step 4 – Recollect results of testing at a central point

If locally executed A/B testing helps local decision-making, how can you increase the benefit for the entire organization?

At Mars, we’ve been collecting our robust measurement outputs in a central location for years. Our knowledge compounded year on year; today, faster decisions are easier to make.

It’s true, we benefit from our scale, but the scale can also be a blocker. We believe in scale. If one single test returns a positive result, you can start to have some basic confidence. You know you've hit the jackpot when 10 similar tests come back with the same result. That’s the benefit of centralizing insight.

Step 5 – Broadcast insights for application

The benefits of an evidence-based marketing culture go beyond the markets where you experiment with the tests. If the process works well, once consolidation is done at the center, less advanced markets can benefit from the business knowledge by tapping into the central insights.

Broadcasting successes and failures are crucial for any market. The benefits transcend employees, reaching customers and peers.

So what are you waiting for? Start building that marketing experimentation culture for your company.