At Recast, our mission is to build the world’s most rigorous Marketing Mix Modeling (MMM) platform. And this starts right from the very beginning – during our configuration process. If you don’t get this right, your model is set to be wrong from the very beginning.
Our configuration process is a sequence of steps that make sure the model is accurate, stable, and trustworthy. Here’s what that looks like:
1. Deep Dive Business Review
We start by understanding every detail about a client’s marketing channels, business operations, and data. Every business – even every marketing channel – is different and it’s critically important to understand those differences to get the model right.
Our team of data scientists, marketing scientists, and model builders work directly with the client to gather detailed information about their marketing activities. For example, we ask questions such as:
- How does the influencer program work?
- Are there specific promotions tied to specific marketing channels?
- How does pricing fluctuate over time?
- How have different marketing channels evolved over time?
2. Initial Model Configuration
With those insights, we configure the model to reflect the client’s unique marketing setup. Our goal is to make sure the model configuration reflects the unique nuances of the client’s business – there’s no standard template we use over and over again.
We think of this as a joint effort. It’s not like we go into a cave and come out with a model on the other side. We work closely with the client’s marketing team to make sure the model fits their unique marketing program.
3. Robustness Checks:
We run multiple checks to make sure the model holds up, such as:
Parameter Recovery:
This involves simulating data where we know the true relationships and then testing the model to see if it can accurately recover those relationships. If the model can’t recover known parameters in a controlled environment, that’s a red flag — something’s wrong with the configuration.
This is a method that most MMM vendors don’t talk about, but it’s a critical part of making sure your model is actually working.
Stability Checks:
These checks are designed to test how well the model holds up under different conditions. We run the model on slightly different slices of the data or with minor tweaks to assumptions and then compare the results.
If the model produces wildly different outcomes based on small changes, it’s a sign that something is wrong and the results are fragile.
Out-of-Sample Forecast Accuracy:
Once the model has passed the initial validation checks, we test its ability to predict the future – which is the ultimate goal of MMM.
In this step, we hold out a portion of the data from the training process and ask the model to predict what will happen in that holdout period – did it pick up true causal relationships?
4. Iterative Refinement
When these checks fail (and they often do), we revisit the model configuration, adjust it, and run the tests again.
We might need to adjust how certain marketing channels are modeled, especially if they are small or highly correlated with other channels. In some cases, we group correlated channels together to improve the model’s accuracy. We also work closely with the client to understand if there’s additional context or information we need to incorporate.
It often takes several cycles of testing, refining, and reconfiguring the model before we get it to the point where we’re confident in its accuracy.
5. Final Validation:
Only after the model has passed all validation checks—parameter recovery, robustness, and out-of-sample accuracy—do we recommend that the client begin using it for decision-making.
Now, our process is very, very rigorous – and we wouldn’t want it any other way. Sometimes new vendors come into the space and claim they can do this in just a few hours… well, good luck.
You just can’t possibly be doing all the checks you need and you’re setting yourself up for a world of hurt. In the end, the rigor of our configuration process is what sets Recast apart—and it’s why brands can trust the insights they receive from our platform.
So take the time here, invest in getting the model right, and actually build the foundation of a successful MMM.
