Media mix modeling at the enterprise level can be quite complex but, when done well, has a huge impact across the whole organization.
Here are 5 lessons we’ve learned to help make MMM work at the enterprise:
1. At the enterprise level, you need one central point of contact who is in charge of making MMM work for that organization.
That person needs to be senior enough that they can get the right people in the room and have enough “political clout” to convince them of the importance of going to these different meetings and taking these different actions. Generally, they also need to have a background in both marketing and data science – either a really technical marketer or a really marketing-heavy data science person.
2. All the different teams in the organization need to be educated and onboarded in a way that makes sense for them.
The finance team needs to be introduced to the different assumptions that are being made and the forecasting capabilities, and they need to understand the uncertainty that comes out of the model. They don’t need to understand the internals of data science and statistics, but they do need to understand how they are going to use this for the budgets that they’re building for next year.
The marketing team needs to be educated on how to take action based on the model’s results. This is a place where sometimes MMM projects can fall short: marketers see a bunch of pretty graphs but they’re unsure what to do with that information. Great onboarding is crucial in helping them make the connections between what they’re seeing in the MMM and the actions that they need to take.
Finally, the data science team needs to understand what data the MMM is using and then be there to help the rest of the organization interpret and use the results.
That triumvirate of stakeholders is what can make a media mix model really impactful and actually useful in an organization.
3. Change management is complex and requires education and re-training.
The most common reason behind an incrementality, experimentation or MMM project failing when they’re being rolled out is that the organization isn’t ready for them and the groundwork hasn’t been properly laid.
We see this a lot, especially in businesses where the leadership team has been trained, often by marketers, to look at GA or their BI tool for a last-touch attribution number to measure the effectiveness of their marketing spend.
The problem is that these other methods –experimentation, incrementality, MMM– aren’t going to match what they’ve been looking at in the past — in large part because they were wrong both in terms of what they were measuring and of what the business should care about.
And that can be shocking to a leadership team that isn’t prepared for it. No one wants to hear that they might have been wasting money and that there’s actually a lot more uncertainty in how their business works than they thought before.
And so, if you’re a marketer in that world, you need to start by educating and walking the executive team through the limitations of their current measurement methods. Instead, take the exec team on a journey toward incrementality. Show them the new methods you want to try, why they’re better, how they’re different, what you expect to happen, and what the process will look like. It won’t be as much of a shock to them, and they’re more likely to be open to it.
Change management is definitely one of the hurdles to successfully rolling out MMM and incrementality, but it’s solvable.
4. Enterprise teams tend to undervalue how complex it is to build MMM in-house.
Real MMM is not as simple as just running your data through a model and getting some output on the other side.
Any data scientist will tell you that the initial build is the fun part. But it’s only the beginning.
A good MMM needs to be updated and maintained over time, and it requires a lot of engineering work to make it stable and consistent.
If you’re an enterprise and building this in-house, you really need to be thinking about that full process. And that comes with tons of questions:
• What’s the data ingestion process and data validation process going to be?
• Who is going to own the model and make sure it’s updated?
• Which servers or where in the cloud will it live?
• Who’s going to be making sure that all of the servers are security-patched and up to date?
• Once the results are generated, how are marketers going to consume them?
• Is that going to get fed into the dashboard, into our data warehouse, where?
• Who’s going to help the marketers make sure that they understand what the results say?
All of those different pieces are required in order to make the MMM project actually valuable. Just running a model in a Jupyter Notebook isn’t going to create very much value for the business.
So if you’re embarking on the MMM journey, make sure you have all of the resources ready to work on building each of those different parts so that all of the hard modeling work that you do actually becomes useful.
5. If you work with an MMM vendor, make sure they are built to handle enterprise MMM.
We’re seeing so many enterprise companies trying in-house MMM first, realizing how much work it actually is, and then moving to work with a vendor like Recast.
We built Recast from the ground up to handle the complexity of the enterprise.
Robyn and Meridian are interesting open-source projects, but you need a very flexible modeling platform to actually adapt the model to the intricacies of businesses at scale.
And it’s not just the modeling framework that I’m talking about here. You need a whole new engineering infrastructure that has to be built in order for an MMM to be useful for large enterprises.
Features like:
• Automated data pipelines
• Automated data validation and alerting
• Tools for the different types of users within an enterprise to help them make the MMM project a success:
— Specific tools for the data engineering team
— Specific tools for the marketing science team
— Specific tools for marketers and planners
• Dynamic report-building tools
• Automated model validation and alerting
• Single sign-on
All of those different pieces are actually really important once you want to operationalize an MMM for the enterprise, and these are just a few of the features we had to build to make Recast work at scale.
A simple out-of-the-box tool that’s designed for one data scientist to use can be useful, but it’s a very different type of platform and actually requires huge amounts of work to operationalize.
Complex businesses with lots of moving parts require models that are built with those challenges in mind.
