In-housing your Marketing Mix Modeling (MMM) can seem like it gives you more control over your data and insights, but building and maintaining an MMM in-house is far more complex than many organizations anticipate – seriously, we know from personal experience.
If you think it’s the right move for your organization – or if you’re, at least, considering it – we want to share the most common challenges you might face along the way and should be looking out for.
1. The Complexity of Causal Inference
One of the biggest misconceptions about MMM is that it’s simply another data science project – you can’t approach MMM with the same mindset you would for a churn prediction model or customer segmentation analysis.
MMM isn’t just about prediction—it’s about causal inference. Instead of just predicting outcomes, the model needs to determine the actual causal effect of each marketing channel on business results.
This distinction is important because it means there’s no straightforward way to determine if your model is right or wrong. There are infinite models that will fit your data just as well, but most of them will be wrong. The challenge is in identifying the rare model that provides the correct causal relationships between marketing channels and outcomes.
MMM is hard, and it’s worse to do it wrong than to not do it all.
2. The Expertise You’ll Need
To build an effective MMM in-house, you’ll need more than just general data science expertise. MMMs that actually work have specialized knowledge in multiple areas, such as:
- Marketing science: to understand the nuances of how different channels interact and influence consumer behavior.
- Causal inference: to have experience in observational causal inference and cause-and-effect relationships between marketing efforts and business outcomes.
- Model evaluation: to determine whether your model is capturing the right relationships and not just overfitting the data, and to know how to execute from the model’s insights.
You can have talented data scientists skilled in machine learning and prediction but they might still lack experience in marketing science, which can lead to well-fitted models with poor inferences – causing marketing teams to make decisions that could cost millions in wasted spend.
3. Maintaining the Model Over Time
Building the initial MMM model is only the beginning of the process. For the model to stay valuable over time, it needs regular updates, ongoing maintenance, and careful monitoring – it’s easy to underestimate the engineering work required to keep an MMM operational.
A successful MMM project will need your attention across multiple stages:
- Data ingestion and validation: You’ll need a solid process to make sure that incoming data is clean, accurate, and up to date.
- Model ownership: There needs to be a clear owner responsible for updating the model as new data becomes available and making sure it’s accurate over time.
- Infrastructure management: The model will require computing power, storage, and regular security updates. Without proper management of the servers or cloud infrastructure, your model’s performance will suffer.
Simply running a model once in a Jupyter Notebook won’t do it. You need to invest in the full data pipeline and infrastructure needed to maintain the model over time.
4. The Continuous Need for Updates and Iteration
Following on from the point above, media mix models are not static. Consumer behavior, competitive environments, and marketing strategies are always changing, and your MMM needs to be flexible enough to adapt.
However, one of the biggest challenges in maintaining an in-house MMM is the continuous effort required to update the model with new data and insights.
An effective MMM requires consistent updates and iteration to reflect changing business conditions and new marketing channels. Many businesses that try in-housing MMM eventually turn to external vendors because of the sheer volume of work needed to keep the model accurate and relevant over time.
5. Navigating Organizational Trust and Change Management
One of the hardest parts of in-housing MMM is not the technical work—it’s building trust and buy-in across the organization. We’ve seen many instances where internal teams build a model, but it never gains traction internally. The internal data team may have worked on the model, but no one internally trusts it, and it never gets used.
This issue becomes even more complicated if you’ve already presented results from your in-house MMM to senior leadership or the board. Bringing in an external vendor at this stage can feel like an admission of failure, especially if the new model provides different insights.
Successful transitions from in-house to vendor-led MMM aren’t easy and need clear communication about the reasons for the change and validation processes to demonstrate the accuracy of the new model.
6. Avoiding Analyst Bias
One of the risks of in-housing MMM is the potential for analyst bias to influence the results. In many traditional approaches to MMM, the analyst has a significant amount of control over which variables are included, how the data is transformed, and how the model is structured.
All of those assumptions have a huge impact on the final results, and the analyst could choose the model that has the results they like. This bias can be unconscious, but it’s still dangerous.
To be clear, this can still happen with a vendor. However, a good vendor should have established clear ways for you to validate the model. This can be easily ignored in in-house models.
7. Handling Cross-Functional Collaboration
Finally, MMM needs cross-functional collaboration across multiple departments—data science, marketing, finance, and operations. Marketers need to understand the results, finance teams need to trust the ROI estimates, and data scientists need to keep the model running smoothly.
Building this “connective tissue” between departments can be a challenge for many organizations, but it’s essential that the insights that come from the MMM actually lead to actionable business changes.
Ideally, this is where external vendors can provide additional value by facilitating communication and aligning the technical aspects of MMM with business objectives.
Final Thoughts
We’re absolutely not saying in-housing MMM can’t work. But we’ve seen too many times how easy it is to invest significant time and man-hours into it just to realize that it’s not the best option for the organization and then have to scratch it and start over again with a vendor.
We hope this article showed you the full scope of work that will have to go into building your MMM in-house so you can make a more informed choice.
As a final note, one common issue we see is the reluctance of internal teams to use external tools due to a “not-invented-here” mentality. However, the most successful teams understand that it’s not about where the model is built, but how well it serves the business. The best teams want to use the best tool for the job, whether they build it internally or work with an outside vendor.
