Nearly every business has an annual cycle: seasons where the business waxes or wanes, or specific dates like Cyber Monday, Memorial Day, or Mother’s Day, which see enormous spikes in sales.
Imagine you have a business that sells hot wings — when do you think the busiest time of the year would be? Football season is going to be relatively busy, ramping up towards playoffs with your biggest night being the Super Bowl. Then once football season ends you probably have a lull until it starts back up again.
This means your hot wings business has seasonality. There’s a predictable pattern to when you’ll sell the most hot wings year in and year out.
If you’re smart, when you’re making your your marketing plan, you’re going to want to take advantage of this seasonality — you want to advertise your wings when people have wings on their minds: when they’re most likely to purchase. That is, you’ll probably increase your advertising budget during football season and ramp up spend during the playoffs leading up to the Super Bowl.
While this is a great strategy for a business, it makes an analyst’s job harder: how do we pick apart the difference between sales that were driven by it being football season (the “seasonality”) and the sales that were driven by the additional marketing? A statistician might say that increased marketing spend is correlated with the natural seasonality in the business.
Unfortunately, this correlation is highly problematic for analyzing the impact of marketing on sales. Many people will tell you that you need to “control for” seasonality when doing a marketing-mix-model, but this would be wrong.
How most MMMs incorporate seasonality (i.e. the wrong way)
To see why it’s wrong, we first need to understand how analysts typically include seasonality in their models. The typical approach is described perfectly in this post explaining how these variables are incorporated in Prophet, the forecasting library from Facebook, used in their open-source MMM tool Robyn. Each day of each cycle — for example, the day-of-week, or day-of-year — is given its own effect. This allows the analyst to draw a curve and say “this is the effect of seasonality on our business”, and then add that curve to the other components of their model.
While this makes a lot of intuitive sense, and is totally appropriate for pure forecasting (which is Prophet’s use-case), it is unfortunately completely incorrect when it comes to media mix modeling. Here’s why:
💡 Your marketing performs better when demand is high.
As obvious as this sounds, it’s not what the traditional approach to seasonality assumes. When you incorporate seasonality in your model, you’re attempting to estimate how marketing would perform independent of seasonal changes in demand. You get results that show how much of sales was due to seasonality, and separately how much was driven by marketing. The function looks like this:
Sales = Base Demand + Marketing + Seasonality.
The problem is marketing is never independent of seasonality: your marketing performs better when market demand is high, and falls flat when interest is low. You’re getting an answer to the wrong question: you don’t care how marketing might work in a world without the Super Bowl, you want to know how much to spend on game night!
How most models give exactly the wrong advice
It gets worse. Controlling for seasonality in your analysis will give all the credit to the season, which means you’ll under-credit the impact of marketing on your sales. Your model would tell you exactly the wrong thing: that you should spend less when people are most likely to buy!
This isn’t an easy problem to solve. We know, because we work with a lot of retailers where seasonality is a big deal, like Away, Harry’s and Mockingbird. Our approach is to model seasonality but not control for it.
It’s no use for the marketer selling hot wings after the end of the Football season, to be told that “our advertising is doing awesome — we’re only losing money because it’s March”. So should you spend more on advertising or not? Of course you shouldn’t, but nothing in your model is telling you that. You’d like to be told instead that “our advertising is doing poorly, because it’s March, so we should cut spend 50%”. What’s happening (March is not a good month for this business) and what to do about it (spend 50% less on advertising) are both clear.
How can Recast model seasonality, but not control for it?
The most important reason Recast can do this, is that we model an ROI for every channel, for every day. Most media mix models (including the popular open-source package Robyn) only model a static ROI, which is the average marketing impact across all time periods. It will tell the hot wing-seller that their ads are just as effective on game day as they are the day after.
To allow model coefficients to change with the seasons requires a more advanced approach. A full treatment is quite technical and beyond the scope of this post, but suffice it to say that Recast’s model incorporates two assumptions into a Gaussian Process prior: today’s ROI should be similar to yesterday’s (which enforces smooth changes over time), and that today’s ROI should be similar to last year’s ROI on the same date (which allows for seasonal cycles). We’re able to do that, because we use a Bayesian approach, which is flexible enough to let us incorporate prior knowledge about each parameter.
So how can you account for seasonality without an advanced model?
Properly incorporating seasonal effects into a media mix model is quite challenging, because off-the-shelf approaches will typically simply “control” for seasonality and output static ROIs. More advanced approaches like state-space models can output dynamic ROIs, but bring their own challenges, starting with simple but common annoyances like enforcing non-negativity.
Outside of working with a vendor that uses Bayesian MMM like Recast, the most important thing that the marketer can do is to understand how their MMM is being specified, and to perform the necessary adjustments in their interpretation of the model. “Our advertising is doing great — controlling for the fact that it’s a slow month” should probably be translated to something more like “it’s a slow month and our advertising isn’t doing that well because of that”. Never view the effects of a model in isolation: take all variables into account when simulating different future scenarios to stay out of trouble.
