Nothing Is 100% Certain in Marketing
Nothing is 100% certain in marketing. Every attempt to understand the consumer is a game of probabilities.
Nothing is 100% certain in research. Every insight carries a probability of it being true, or wrong.
(Almost) nothing is 100% certain in life. Learning the basics of probabilities will certainly make you a better marketer.
I started in Mars 10 years ago, and I was lucky to join a “marketing science” team, a team that puts sciences to the benefit of marketing. I was a graduate engineer, so their assumption that statistics was one of my favorite classes in university was almost inevitable. But it was so wrong. To their surprise, I had little recollection of anything related to p-values, confidence intervals, or sampling methods. So I had to learn everything from scratch, on the job. This changed how I see marketing and ultimately how I acquire information about the world.
The shocking truth is that no consumer research can tell you with 100% accuracy what happened simply because we live in a world of probabilities. In most circumstances, every insight has a 20% chance of being completely WRONG. Yes, it is 80% right, but 20% wrong. And that’s almost always enough to make marketing decisions. Please take a minute to breathe and think about that. Remember when we told you that Facebook Stories work or that Display Ads don’t work, how confident were you? Probably 100%. In reality, you should’ve been 80%.
Most researchers keep this information away from the marketing department. I think the opposite.
We should talk openly about confidence intervals because fooling each other with precise numbers alone doesn’t work.
Another example is related to the variability of the outcome. When we tell you the ROI of TV is 1.23$, the ROI is probably NOT 1.23$. That’s just the center of the interval of confidence. We should tell you it is somewhere between 1.12$ and 1.34$. As researchers, we must keep that interval as small as possible. We do that by increasing the sample size. But in any case, in 20% of instances, the result is outside of this interval. So not only that we give you a false perception of preciseness, but we are certainly wrong about it.
You might be angry and think you need the precise number, but can you not make a decent business call with an interval of numbers?
My advice would be to always be curious. Every time you receive a piece of information, ask yourself three questions (or more):
- How big is the sample size of the research?
- What is the confidence interval of the results?
- What is the confidence level of the outcome?
If you have these answers, you will make better sense of the result and avoid stupid generalizations.
