Last updated: August, 2024
At Recast, we hold strong views about the right way to do Media Mix Modeling (MMM) that can be surprising to teams accustomed to outdated methodologies. Specifically, we believe that in order to be valuable, a Media Mix Model must be:
- Verifiable
- Timely
- Actionable
This article explains how these three interlocking features are critical to actually getting value out of a Media Mix Model, and why Recast was built from the ground up to reflect these values directly in our platform.
1) Verifiable
Verifiability is the most important feature of a Media Mix Model (MMM).
MMMs are complex, assumption-driven statistical models. The complexity and the range of different plausible assumptions someone can make when running a Media Mix Model makes them highly likely to be wrong.
And not just a little bit wrong, but totally and completely wrong. If you’re using an MMM for decision-making and budgeting, when the MMM is wrong it means you are wasting your marketing budget – for many companies this can amount to millions of dollars in wasted ad spend every year!
When you’re building or using an MMM, the most important feature, by far, is the ability to validate that the model’s results are actually correct and can be trusted for decision-making.
If you cannot validate that the model’s results are correct, then the UI and planning tools that are wrapped around it do not matter. Correctness must come first.
There are two main ways to validate Media Mix Models:
- With out-of-sample forecast accuracy checks. That is, by having the MMM forecast data it wasn’t trained on and comparing those results to actuals.
- With experiments like GeoLift tests or channel go-dark tests that can generate causal estimates with fewer assumptions required.
Media Mix Models should place both of these validation methods front and center so that the users of the tools (marketers, planners, analysts) can evaluate the model’s performance themselves and know if they can actually trust its results.
2) Timely
Media Mix Models are decision-making tools. A report on how your marketing performed two years ago will not be very helpful for helping you make budget-allocation decisions today.
The more timely an MMM’s results are, the more useful they are for decision-making because the results will more closely match current marketing performance and therefore will be most helpful for the budget allocation decisions marketers need to make today.
Beyond just decision-making relevance, a more timely MMM is also more verifiable because the MMM can generate falsifiable predictions about what will happen in the near future. If the MMM only covers a period of time greater than 3 months in the past, it’s very difficult to prove the model wrong. It’s effectively untestable.
But, if the model is being refreshed every week (like how most brands use Recast), then the model is always up to date and therefore always testable. For many MMM vendors this is scary. They don’t want their models to be testable, because if the model is testable it can be proven wrong. But we believe this is the wrong approach: if you want the model to be correct, you must be able to test it and therefore the results need to be timely enough to be testable.
3) Actionable
If a tree falls in a forest and no one is around to hear it, does it really make a sound? If you went through all of the trouble of building a Media Mix Model and no one is able to take action on it, what was the point?
Media Mix Models should help you manage your marketing program more efficiently. They should help you plan better and reallocate your marketing budgets more dynamically. They should help your business earn more profit through more efficient marketing.
It’s critical that your Media Mix Model not just be a static report in a powerpoint deck, but rather a set of tools that help you:
- Perform scenario analysis of what may happen in the future
- Optimize marketing budgets subject to real-world business constraints
- Track your pacing against your business goals and marketing plan
The Media Mix Model should be a tool that you use regularly to help you manage your marketing program, not just a set of static charts with interesting insights.
Conclusion
We built Recast from the ground up to tackle these problems head-on. Recast is built to be fully verifiable, update fully automatically (on a weekly cadence) and to be an actionable decision-making and marketing planning platform.
We believe that this is the future of marketing measurement and settling for anything less in a Media Mix Model means setting up your organization for failure.
