We just shipped multichannel impact tests – a new way to incorporate evidence from experiments where multiple channels moved at once into your marketing mix model.
Often, a team will run an incrementality test that simultaneously increases the budget on two paid media channels (like testing both YouTube and TV together) in some cities without turning anything off elsewhere.
Say the test comes back at 5x ROI, plus or minus 1.2x. That’s the combined return from YouTube and TV. This number is consistent with both channels returning 5x. But it’s also consistent with YouTube at 6x and TV at 4x. The single result can’t tell you which.
Most MMMs expect changes in spend in a single channel per incrementality test. That means a joint YouTube + TV result needs to be forced into that shape before the model can use it. But this requires judgment calls about how to split credit between the two channels – baking your assumptions into the experiment results as if they were data and diluting the causal signal you worked hard to collect.
We’ve heard from many teams running exactly these kinds of tests – either because their agency buys channels as a bundle, because their media plan links their flighting, or because running isolated channel-by-channel experiments is prohibitively expensive.
Now, with multichannel impact tests, there is no need to compromise when using these tests to calibrate your MMM. Recast takes the results and design of a multichannel test and uses them as experimental ground truth to anchor your model.
Who benefits and how to access
Teams in the following situations will benefit from this feature:
- Your experiment varied spend levels, and your team wants to know: did that joint spend increase we pushed actually work – and can my model learn from what we saw?
- You don’t want your experiment results simplified just to fit the model. If the test was multichannel, the calibration should be too.
Multichannel impact tests are available now for all Recast customers – just reach out to your dedicated marketing data scientist to learn more about incorporating them. Calibration is a key component of model trust, and we’re excited to see more teams leverage this development.
