There are thousands of non-marketing factors that can make an impact on your company’s base sales. Consumer sentiment, GDP growth, inflation, consumer saving rates, interest rates… we could list dozens and dozens of exogenous factors that can change your base sales.
But when it comes to marketing mix modeling, we really need to think about what’s the relationship between those exogenous factors and, most importantly, what’s the true incremental ROI of marketing investments.
And the problem is that a lot of these exogenous factors interact with our marketing spend. And so it’s not that changes in interest rates or consumer savings rates or inflation, choose your pick, only impact our baseline sales – they also impact the effectiveness of our marketing spend.
And so, if we control for them naively, we will get biased estimates for these other variables that we actually care about. And so it’s really important to be very thoughtful about exactly what we are controlling for and how that is going to impact the results that we care about.
So, should you control for exogenous factors in MMM?
Our point of view is that trying to control for all of those external variables is a fool’s errand. We think that’s misguided for a couple of different reasons:
1 – There are a ton of things that impact every business, and there’s no possible way that any statistical model can control for all of them.
If your ability to generate causal inferences is predicated on you controlling for all these factors, you’ve already failed. It’s impossible.
Here’s a clear example: you want to use your MMM to forecast and plan into the future. But if your model needs you to input variables, then you need to actually use those same input variables when you’re doing a forecast – and how are we ever going to do that?
It’s impossible to know how much inflation is going to be a year from now, or how interest rates will change – so we don’t want to include those variables in the model because we won’t be able to forecast for any of them.
(If we could forecast all those variables correctly, we wouldn’t do MMM – we would go make a bajillion dollars in the stock market).
2 – Model building that relies on controlling for all factors has far too many degrees of freedom for its analysts.
Whoever is building the model gets to decide what is included and what is not, and that tends to create a lot of room for bias. You end up building a bunch of models that don’t work, but show your CMO the one they want to see – it’s a terrible modeling practice.
As a general rule, you should watch out for any process where there’s a lot of judgment involved from an analyst. Whoever manages your marketing mix model should not have the power to arbitrarily shape the results.
I’m sure these bad practices stem from the desire to be helpful, but they tend to lead to really bad decision-making processes. Your MMM should not be a cover-your-ass product. It should be unbiased and based on ground truth.
How we model exogenous factors at Recast:
Covid is a really great example to illustrate how we think about this:
Recast has some customers who were really negatively impacted by Covid, and their sales went down. We also had some customers who were really positively impacted by Covid, and their sales went up.
One effect of Covid was on the organic side of their business – some companies had an increased organic demand, others lower. The other effect was on the effectiveness of their marketing spend – every additional dollar was more effective for some companies because people were more primed for purchase, and the opposite happened for other companies.
Because Recast is a time series model at its core, and we’re estimating how the ROI for marketing channels are changing over time and how the organic demand for a business is changing over time, we don’t need to control for Covid because we’ll see those effects show up through those underlying variables.
If we saw a company’s organic business do better and their paid performance gets better, we didn’t have to say it was due to Covid, we just wanted it to be actionable – they should spend more money on marketing because they were getting that additional better performance.
So we treat these exogenous factors the same way we treat seasonality: we design a very flexible model that doesn’t rely on getting causal inference by having to control for every possible factor that could potentially impact a business, and use a time series model that evolves and learns how baseline sales change over time.
The model doesn’t know the underlying cause for the baseline change, but it doesn’t have to. It’s flexible enough to pick that up by itself.
For the majority of companies, we think that’s a really appropriate model that becomes more actionable, and focused, and doesn’t induce potential sources of bias.
TLDR:
- Non-marketing factors like GDP growth and inflation not only significantly impact base sales but also interact with marketing spend, adding layers of complexity to accurately measure the true incremental ROI of marketing investments in MMM.
- Attempting to control for all external variables in MMM is impractical and can lead to biased estimates. In the vast majority of cases, we recommend not to do this.
- By utilizing a flexible, time series model, Recast adapts to changes in organic demand and marketing effectiveness without the need for controlling every exogenous factor, offering a more actionable and unbiased analysis.
