“Storytelling with data” sounds great in theory – we all would love to have a single source of truth that always outputs one clear, actionable insight with no uncertainty. But in practice, most real-world data doesn’t work that way. Especially in media mix modeling and incrementality work, the outputs are […]
How to Communicate Data So It Drives Better Business Decisions Read More »
If you’re treating p < 0.05 as the litmus test for your marketing experiments, it’s going to be really hard to build a solid measurement program. Many marketers still blindly follow this rule: But a p-value doesn’t tell you whether your campaign worked, how big the effect was, or whether
What to Do When Your Marketing Experiment Isn’t Statistically Significant Read More »
Every marketing measurement strategy comes with a cost. Whether you’re running incrementality tests, MMMs, or lift studies — there’s always a tradeoff. Andrew Covato calls this the “measurement tax” — a concept he shared in our Recast Marketing Measurement Coffee Break. This tax comes in two forms: explicit and implicit.
Why Marketing Measurement Always Comes with a “Tax” and How to Pay It Smartly Read More »
Marketers love to talk about experiments. And experiments are great, but marketers who only focus on A/B tests and holdout studies are missing out on levers that can move the needle for their business. So here’s our premise: the sharpest marketing organizations need to think in bets instead. What is
How to Prioritize Marketing Bets for Maximum Incremental Return Read More »
There’s a lot of confusion from marketers about the right way to use lift tests or incrementality experiments to calibrate the results of a media mix model (MMM). We see questions like: You need to be thoughtful when incorporating information from incrementality experiments into an MMM, so in this post
How to Use Lift Tests to Calibrate an MMM Read More »
Most marketers want to test everything. But you don’t have unlimited budget, time, or team capacity. So you need to be ruthless about prioritizing what actually gets tested. That means knowing where a test will generate the most value — not just statistically, but strategically. Here’s how we help the
How to Prioritize Experiments When You Can’t Run Them All Read More »
Imagine a brand runs a geo lift test. They shut off ads in three regions for a few weeks, compare the results to three control regions, and observe a lift in sales in the treated geos. The model spits out a stat sig result, and the team gets excited. The
A Framework for Evaluating the Signal Quality of Incrementality Tests Read More »
“Incrementality testing” is extremely hot right now. And while I am personally extremely enthusiastic about this type of experimentation and I am very glad to see that more organizations are running more experiments, I see a lot of confusion around important concepts regarding these types of experiments that I think
Fact and Fiction in Marketing Experimentation Read More »
Setting the scene Marketers have been given an impossible task. They are being asked to grow the business in the face of a shifting landscape and terrifying amounts of uncertainty. CEOs and CFOs want certainty and an inexorable march of growth, but they don’t give marketing teams the tools and
Introduction: How to Build an Incrementality System Read More »
If your executive and finance team has become addicted to only looking at the last-touch attribution report in Google Analytics, this can be a real problem. It’s critical to help them understand the nuances of solid marketing measurement beyond just last-touch reporting. We’ve seen it too many times: If the
How to Help Your Finance Team Understand the Value of Incrementality Read More »
