Marketing/Growth

Hypothesis before evidence: how to build an action thesis in marketing

Data without a question confirms anything. Why the hypothesis has to come before the evidence, how to write one you can refute and how to combine several into a thesis that tells you where to invest.

Murilo Anzini 6 min read

A sentence I hear a lot: "let's run the campaign and see what the data says". I get the intention, but in my view, data doesn't say anything on its own. It answers questions. And if you didn't ask the question first, any answer will do.

With enough slicing, there's always an audience that converted better, a time slot that outperformed, a creative that stood out. It looks like discovery, but much of it is noise that found a story after the fact. That's why I defend a simple order: hypothesis first, evidence second.

Why the order matters

When the hypothesis comes first, it does three things evidence alone can't:

  • It gives direction to data collection. You know which events you need to track, which audience to compare and how long to wait. Without that, you measure whatever the tool gives you by default (clicks, CTR) and try to draw conclusions from shallow data.
  • It defines what being wrong would look like. If I write down beforehand what I expect to see, I can recognize when I didn't see it. If I look first, I can always explain the result, whatever it is.
  • It protects you from your own cheering. Everyone wants the idea to work. Writing the hypothesis first is a way to tie your hands so you can't move the goalposts later.

The idea isn't new. Karl Popper argued that what makes a claim scientific is that it can be refuted. In marketing, the practical translation is: if no possible result would make you change your mind, you don't have a hypothesis, you have a conviction.

A practice I use before any research

In my ebook I describe how I prepare for customer interviews: I put together a document with the points I want to learn, write the base questions and answer them as if I were the customer, before talking to anyone. Then I compare with the real answers.

This exercise looks silly, but it's exactly hypothesis before evidence. My answer is the hypothesis. The customer's answer is the evidence. And the gap between the two is where the learning lives. When I don't write it down first, I tend to hear the customer's answer and think "right, I figured as much", which is almost never true.

A hypothesis is not a hunch

A hunch is "I think young people will like it". A hypothesis has structure. The format I usually use:

We believe that audience will behavior because reason. If that's true, we'll see measurable signal within time frame. If we don't see threshold, we drop it or rethink it.

The "because" is the most important part and the most forgotten. It's what tells you what you'll do if the hypothesis holds. Without the reason, a positive result teaches nothing: you know it worked, but not why, and you can't repeat it.

And the threshold has to be written beforehand. "Let's see if it improves" isn't a threshold. Defining the number, the window and the comparison group before you start is what turns a change into an experiment.

From hypothesis to action thesis

A single hypothesis answers a small question. To decide where to invest, I combine several into a thesis: a belief about the market that rests on chained hypotheses, each one testable.

I'll use a hypothetical example to make it concrete. Imagine software for small accounting firms. The thesis could be:

Small firms buy when they realize they'll save time on the month-end close, not because of price.

That thesis depends on a few hypotheses, and each has a cheap test:

HypothesisIf true, we'll see...Cheap test
The main pain is time spent on the closeIt comes up spontaneously in conversationsInterviews without mentioning the topic first
They search for a fix to that, not for "accounting software"Close-related terms have meaningful search volumeKeyword research and Google Trends
A time message beats a price messageThe time ad brings more people who move through the funnelTwo ad versions, same audience and budget
A guided trial converts better than a free oneMore accounts reach their first closeA/B test in onboarding

Notice the order. The first rows cost almost nothing: conversations and research. The last ones require product, campaigns and traffic. Cheap evidence decides whether the expensive kind is worth paying for. If the month-end pain doesn't show up in conversations, I don't even build the campaign. The thesis fell, and it fell cheaply.

That's the principle I've followed since my infoproduct days: test before you spend, take risks with some safety net and, above all, don't lose money.

How the thesis becomes a page, a campaign and a funnel

Something I do, and explain in detail in the ebook, is move the value hypotheses onto the sales page as sequential blocks, one block per value proposition, each with its own event when it's fully on screen. That way, the page funnel stops being "visited and bought" and starts showing at which hypothesis the person gave up.

That's only possible because the hypothesis came first. If I had built the page and only then gone to look at analytics, I'd have the default events and no way of knowing which argument was working.

The precautions that matter most

  • One hypothesis at a time. If I change the page message and the campaign audience in the same week, I can't tell which change made the difference. If the change is on the site, I don't touch the campaign at the same time.
  • Write down what would make you give up. And respect it when it happens. Contrary evidence is worth more than supporting evidence, because it's the kind you least want to see.
  • Watch who is in the data. Evidence can come from a group that doesn't represent your audience. I wrote about that in selection bias in the funnel.
  • Don't confuse liking with paying. There's a big difference between people liking what you offer and being willing to pay for it. Praise in an interview is a weak signal; a purchase is a strong one.
  • Theses age too. What was true with your first customers may not be true with the next ones. Revisiting the thesis is part of the job, not a sign of failure.

A simple ritual

To keep this from staying theoretical, I keep a living table with the theses and each one's hypotheses: what we believe, what we expect to see, the status (open, confirmed, refuted), the evidence we have and the next bet. Every week or two, we look at the table together and decide what to test next.

It's a small thing, but it changes the conversation. Instead of "I think we should invest in video", the discussion becomes "which hypothesis of the thesis does video test, and what do we do if it fails?".

Summary

Data without a question confirms anything. Write the hypothesis before looking at the evidence, with the reason, the expected signal and the threshold that would make you give up. Combine hypotheses into a thesis that explains where to invest, test the cheap things first and only pay for expensive evidence when the cheap kind says it's worth it. One change at a time, respect for contrary evidence and patience to revise. That's how an opinion becomes an action thesis.

#hypothesis#growth#experimentation#strategy#analytics

Murilo Anzini

Hands-on writing on engineering, product, data and marketing for people who want to understand, and build, the whole journey of a product.

In times when everything comes from AI, I like to write because it reinforces my learning and it's my moment of personal growth, and sharing it with you is a pleasure. The ideas, opinions and stories are mine; AI helps me structure, review and translate, because I'm only human.

About me

Free ebook

Criando infoprodutos de valor

The method I use to bring product, marketing and sales together when creating digital products: the way startups test hypotheses, understand needs and build what the market wants to buy.

  • Understand the customer and set the initial hypotheses
  • Build and test a lean MVP
  • Measure progress and validate with data
  • Know when and how to pivot
Download for freePDF · 23 pages · in Portuguese · no sign-up