Is it possible to have a multi-channel, multi-market campaign and still get actionable insight from your media mix modeling on channel efficacy?
Or does a brand have to temper go-to-market speed + aggressiveness to best set the table for long-term insight?
Our co-founder Michael just got this great question last week, and here’s what he replied:
MMM requires variation in order to generate incrementality estimates, and the more variation the better. The quality of the read from the model is generally determined by the total amount of variation in the marketing activity over the time period being modeled.
In the event that some channels are perfectly correlated (TV and Radio follow the exact same spend pattern at different scales, for example), then no statistical model will not be able to differentiate between them – statisticians call this problem multicollinearity.
The multicollinearity problem in marketing:
This is a very common problem: marketing teams often up or lower their budget across all channels at the same time. It’s also very common with seasonality.
Especially in multi-channel, multi-market campaigns, marketers will often increase or pull back spend in similar channels by the same amount – just because it’s easier to do at scale. When things go well, we increase spend on all our channels simultaneously. And when things don’t go well, we pull back on all of them too.
For example, if we get some extra budget, we might increase radio and TV spend both by 10%. If we then need to determine the effectiveness of radio versus TV advertising, we will run into collinearity and we won’t be able to discern if it’s TV or radio that is incremental.
One preliminary method to detect multicollinearity is examining the correlation coefficients among the predictor variables. A correlation coefficient of 0.8 or above typically indicates high multicollinearity – which will increase the variance of the coefficient estimates, leading to wider confidence intervals or less precise estimates.
Solutions to break multicollinearity and find signal for your MMM:
What you don’t want to do is put it in a traditional regression-type model. You will have a very hard time solving for this and you might even see results that show a negative impact on marketing spend, which shouldn’t be possible (for >99.9% of situations).
It would be like saying that, if you spent a dollar on this channel, you would get back a negative return for that – that’s (almost) never true and a really big problem.
One modeling solution is to estimate the impact of the two channels together.
For example, you might be able to say “Combined, they have an ROI of 4.5x though we don’t know if they’re both good, or one is bad and the other great, or vice versa”.
Fortunately, all is not lost when it comes to a large, multi-channel campaign. If there are ways to introduce more variation in the spend, you can get more signal.
That could be something like months or weeks of relative pulsing of spend on TV and radio (for example). So it could be that in week 1 you spend relatively a bit more on TV and in week 2 you spend relatively a bit more on radio, and then repeat that pattern throughout the campaign.
This divergence allows the company to observe changes in revenue linked to each specific channel, helping break the correlation and reduce multicollinearity.
Also, it’s worth noting that the multicollinearity problem comes in retrospective analyses. However, marketers have the power to break the correlation moving forward.
I know that this sort of extra variation can be a PITA to implement operationally, but it can give more variation for an MMM to exploit to get more precise estimates of performance.
And, of course, it won’t be a problem to estimate the effect of the campaign as a whole — the extent to which individual channels can be credited will depend on the amount of variation available in the system.
TLDR:
- Is it possible to have a multi-channel, multi-market campaign and still get actionable insight from your media mix modeling on channel efficacy? Yes, as long as you solve for multicollinearity by varying spend between channels.
- Or does a brand have to temper go-to-market speed + aggressiveness to best set the table for long-term insight? Not really. I think really top-tier marketing teams will try to inject as much variation in marketing activity as possible into their plan, subject to the business constraints and the goals of the campaign itself.
