Test, Learn, Fix and Go Again

Marketing loves success stories. We write case studies about the campaigns that exceeded targets, websites that increased sales and creative ideas that generated enormous reach. What tends to receive considerably less attention is the campaign that underperformed, the email nobody opened or the idea customers simply did not care about.

That is unfortunate because those failures often contain some of the most useful information available to us.

Black Box Thinking by Matthew Syed helped reinforce this idea for me. One of the central themes of the book is that high-performing organisations become better by creating systems that allow them to examine mistakes rather than hide them. The principle applies particularly well to marketing because so much of what we do gives us opportunities to learn from real customer behaviour.

We can test messages, subject lines, calls to action, landing pages, creative formats, propositions, audiences, timings and sometimes pricing. Not every decision needs an elaborate formal experiment, but most campaigns should at least create some form of learning. That requires us to have an expectation beforehand rather than simply looking at the numbers afterwards and constructing an explanation that makes us look good.

The cultural side of this is probably more important than the technology. If an organisation treats every disappointing result as somebody's personal failure, people quickly become skilled at defending themselves. They choose flattering metrics, avoid risky ideas and explain away poor results. That might make meetings more comfortable, but it makes the marketing worse.

I would much rather work in an environment where somebody can say that we expected a particular campaign to increase registrations, it did not, the data suggests a particular barrier and we are going to change the next version accordingly. That is not failure in the negative sense. It is an organisation learning.

Experience obviously still matters. After many years working in marketing I have plenty of instincts about what I believe customers might respond to, and those instincts are useful. But twenty years in the industry does not make somebody incapable of being wrong, or just making a cock up. Markets change, people behave unpredictably and sometimes an idea that looks excellent in a meeting simply does not work in the real world, don't let it stop you thinking of the next good idea.

That is one of the reasons I think marketing needs both confidence and humility. We should have enough confidence to make decisions and put ideas into the market, but enough humility to accept the evidence when customers tell us something different.

The objective is not to prove that the marketer's original idea was correct. The objective is to make the marketing better the next time around.