Airbnb famously turned off their paid media and argued it turned out to be a successful decision. But was it? While Airbnb’s decision to turn off paid media may have resulted in some cost savings, it’s important to consider the potential long-term effects on its customer acquisition and brand equity.
It’s not the only time this has happened: P&G turned off $200 million of their digital ad spend, Chase reduced their programmatic reach from 400,000 sites showing its ads to 5,000 sites, Uber turned off $120 million of their ad spend… none of them saw sales decline.
Experimenting with turning off channels can be a good way for brands to audit their marketing channels and test their incrementality – and experimenting with cutting the channel for some time is a great way to maximize information gathering and, if done well, you can minimize your opportunity costs.
In this article, we will cover why and when you should turn off a channel as an experiment, how to execute it, and the potential challenges you need to be aware of before you execute this.
When Should Brands Experiment Turning Off Channels
Turning off a channel as a test can be a generally good exercise, but some specific circumstances and scenarios can help dictate how often and why you should implement this. These are some best practices that we’ve seen can create some urgency:
- If you have at least 2 marketing channels
You probably should not test turning off a channel until you are executing in at least two marketing channels. Most companies fail, so they don’t ever really get to more than one main channel: don’t worry about incrementality until you’ve grown past that. Once you get past two channels, there’s less oversight over the granularity and detail from management – and that’s when you can start missing things that turning off a channel can help discover.
If you’re a CMO and you’re getting reports from multiple channels, you might want to start thinking about testing this. Your channels might start getting less incremental and it’s time to audit them.
- If you’re having multicollinearity problems
One common reason for experimenting with channel cuts is to help solve for multicollinearity, which is when different channels are directly influencing each other’s performance. By turning off a channel, you can isolate the effects of the remaining channels and better understand their individual performance.
Recast is made of statisticians, data scientists, and analysts – we love solving problems through data. But we are also marketers, and we should not forget that media buyers have the power and agency to make changes in their campaigns and move levers to generate variation and noise so you can actually see the effectiveness of each channel. Sometimes you only need statistics if you didn’t run a good enough experiment: if there isn’t a huge spike or dip, the change wasn’t big enough.
- Depending on economic conditions
Macroeconomic conditions can be a good indicator of when it is time to test the incrementality of your channels. As we write this article, we have higher interest rates and funding is harder to get. If you’re a startup and you’re burning a lot of cash, you might look at cutting some of your marketing – but you only want to cut the least incremental channels or slower growth will nullify any cost savings.
- To audit and avoid fraud
In 2017, Uber revealed that it had been overcharged $100 million by a third-party ad network due to fraudulent activity. They decided to turn off the channel and discovered that it wasn’t incremental because they were being charged Uber for ads that were never viewed by real users, but rather by bots and automated scripts. If you’re suspicious of a channel, turning it off as an experiment can be helpful to audit it.
- To stop “inertia” marketing channels
It’s good practice to run these experiments regularly. We often see companies “pick up” channels that they just keep running without anyone questioning why they’re doing it. Once a quarter or once a year, depending on how important the channel is, clean house and automatically run these lift tests to measure their incrementality and make sure you’re allocating your budget efficiently.
How Should Brands Experiment Turning Off Channels
One common question is how long should you turn the channel off to get enough data. There’s not one single answer as it mostly depends on the hypothesis of your channel’s lag.
For example, if you’re testing branded search campaigns on Google Ads, they’ve already learned about you from somewhere else and it’s unlikely to be incremental. Here, you could test by alternating turning it on and off every hour and netting the differences over a couple of weeks.
For longer lag channels, you need to turn it off completely and for longer periods. It can be riskier to do that so, instead, you could turn it off in a specific location and run a geographic experiment. Turn off 10% of the states and see the difference while mitigating the risk of losing sales if the channel is very incremental.
Another key element to making these experiments valuable is having a centralized “marketing data warehouse” to aggregate all data points in one place. This data consolidation is fundamental in providing confidence in accurately measuring the actual impact of marketing efforts on revenue, beyond the perceived impact. The value of the test is in the information you’ll gather, so make sure you’re getting accurate data.
For example, when running paid search on your brand term, Google Ads may report a certain number of sales, while Google Analytics may provide different statistics, incorporating SEO factors. Cross-referencing Shopify revenue can provide a more precise picture of the number of customers that came from that Google ad and made a purchase.
The Opportunity Costs of Cutting a Channel
You should only experiment with completely cutting off a channel if you understand the opportunity costs and potential risks that come with it. Here are some you might face:
- Adstocks
If you turn off a channel and you don’t see any immediate decline in revenue, it might be that the channel was not incremental. Or, it might be that there’s a lag until it is reflected in your performance.
If you went to the gym for 10 years and stopped going for two weeks, you wouldn’t feel different – but you wouldn’t think those 10 years to be a waste of time. In the same way, branding channels take much longer than direct response channels for you to see the consequences of turning them off.
- Is your brand a verb?
One of the dangers of trying to replicate Airbnb’s experiment and expecting the same results is that most brands don’t have the brand equity that they have built over the years. Even though they cut performance marketing, it is much easier to do so when your brand is a verb.
Google, Airbnb, Kleenex… those sorts of brands are operating in a very different environment than the vast majority of brands that are doing advertising. If your brand isn’t a verb, you might want to do this more carefully.
- Pandemic outliers
It’s totally possible though that cutting Airbnb’s spend in non-pandemic times would yield different results. ROI could be low during the pandemic but high at another time. Be careful when imitating similar chess moves when the board now looks completely different.
- Brand bidding and incrementality
While brand bidding can be an effective way to maintain visibility and protect a company’s brand, it is not typically considered an incremental marketing strategy as it does not generate new demand or bring in new customers. Cutting other channels that are often more incremental elevates the risk and opportunity costs.
- PR will fade
Finally, brands must consider the potential impact on PR when cutting a channel. We’ve often seen an initial boost because of the attention that publicly cutting a channel can bring in, but eventually, it may no longer be newsworthy.
Conclusion
Turning off a channel and testing its incrementality can save your team millions of dollars – but it goes beyond that. This idea of experimentation, of testing our assumptions, of auditing our marketing… is a great attitude to have. More marketers need this attitude and more businesses need to be willing to take these sorts of risks.
If you turn off all of your marketing or some of your marketing, you might lose some sales in the short term if the channel really does turn out to be incremental. That being said, the information that you get back from it is incredibly valuable because you’re able to actually learn what is the true incrementality of your marketing dollars. That’s the actual value of the experiment – information.
We highly recommend this at Recast: run experiments, take big bets, take big swings, and see what we can learn from it. And it might be like Airbnb, where you’re actually overspending on marketing right now and you can pull back and save budget. Or, it might go the opposite direction and you realize you’re really dependent on marketing. Either way, the learnings are essential.
