Today, we introduce the Lift Test Calculator – an open-sourced tool designed by Antonio Vargas, a senior data scientist at Recast.
The Lift Test Calculator was created to address the gap in the way lift tests are conducted and interpreted. Antonio and the Recast team recognized that marketers often ran into issues when trying to get a handle on the uncertainty that existed around lift tests.
Traditionally, lift tests are simplified down to two numbers: the revenue from the control group and the revenue from the exposed group. However, this eliminates the ability to account for uncertainty. A crucial variable that was often overlooked was the size of the control group. If the control group was small, then the comparison between it and the exposed group would be less reliable than if they were both substantial in size.
The Lift Test Calculator was designed to allow marketers to input the minimal information required to understand the uncertainty around their final incrementality estimate.
Marketers across industries can leverage this tool, especially when determining which channels to focus their budgets on. By taking into account the uncertainty of each estimate, they can optimize their budget allocation more effectively. If you’re running lift tests on any channel, this is for you.
A Quick Guide: How it Works
The Lift Test Calculator operates on a few basic pieces of information: marketing spend, sizes of the control groups, number of orders for each group, average order value for each group, and the standard deviation of the order value for each group.
Upon feeding these details into the calculator, marketers get a tangible ROI figure accompanied by a standard error for the ROI. The ROI value shows how much return marketers can expect for each dollar spent, while the standard error indicates how reliable this ROI number is.
In contrast to traditional ROI calculators, the Lift Test Calculator goes a step further. Not only does it help calculate ROI, but it also introduces an innovative measure of uncertainty around the incrementality estimate, providing a more nuanced and accurate evaluation of your marketing efforts.
We’ve open-sourced it – just make a copy to the doc below.
https://docs.google.com/spreadsheets/d/1Gt7S6NzEI7U6cswv6xMgclhaiUU3TqjneGWyGjg9G5g/edit?usp=sharing
The Bigger Picture:
At Recast, we believe ecommerce brands need to develop both their testing mindset and muscle, and this tool should be able to help you achieve that in a simple and easy way. If you want to chat about taking it to the next level with MMM, we’re happy to have a conversation.
