If you’ve already run geographic-based incrementality experiments using other tools or methods, you can use GeoLift by Recast –– to reanalyze your historical test data. This can help validate previous results from other vendors or in-house methods, apply more sophisticated synthetic control methods, or simply get a second opinion on your experiment’s incremental impact.
Start your free GeoLift by Recast trial here.
Why Reanalyze Past Incrementality Experiments?
There are a few reasons you might want to reanalyze previous incrementality experiments using GeoLift by Recast:
- Validation: Reanalyzing experiments allows you to compare results across different methodologies and build confidence in them.
- Advanced Methods: GeoLift by Recast allows you to apply synthetic controls that may provide more precise lift estimates than other methods like matched market tests.
- Standardization: Analyze all of your historical incrementality tests with a consistent battle-tested methodology.
When you’re ready to reanalyze a past incrementality experiment, you can follow this step-by-step guide.
Step-by-Step Reanalysis Guide
1. Prepare Your Historical Data
Before uploading data to GeoLift by Recast, ensure your data includes:
- Pre-test period: A full year of data before your experiment ended, 30 of which happen before the test starts (if you don’t have a full year you can still proceed as long as you have at least 90 days)
- Test period: All days when the experiment was active
- Post-test period: Several weeks after the experiment ended (for cooldown analysis)
The CSV that you upload should include three columns:
- Location (geography identifier)
- Date
- KPI (ex. revenue or conversions)
2. Upload and Map Your Data
Next, open GeoLift by Recast and navigate to the Analyze tab. Upload your historical dataset and map your columns appropriately:
- Select which column contains locations
- Select which column contains dates
- Select which column contains your KPI
- Specify your date format
One note on this step: If a geography/date combination is missing, or if the KPI value is missing for a particular geography/date combination, GeoLift by Recast will assume the KPI was 0 on that day.
3. Configure Reanalysis of Your Historical Experiment
Once the data is uploaded, you can configure the analysis of your past incrementality test by entering the following details:
Experiment Timing:
- Start date: When you began the spend change
- End date: When you returned spend to normal
- Cooldown period end: When you choose to stop analyzing the data for additional conversions*
*We recommend choosing an analysis window such that the majority (~95%) of conversions you expect to come from increasing spend (or conversely, not occurring due to decreased spend) have been observed in the historical data. If you’d like to read further, see more in the Configure your Analysis section here.
Test Locations: When uploading test locations, you have two options:
- Type the test locations manually
- Upload an additional CSV with a “locations” column listing all test geos
Experiment Details:
- Select your outcome variable type: Revenue or Conversions
- Select your experiment type: Spend increase or decrease
- Select spend amount: The actual dollar amount you added or removed
There are also a few special considerations you may want to account for at this step. For example, if you know certain control geos may have been contaminated (e.g., received some test impressions due to geographic spillover), exclude them from the analysis.
Choosing the correct cooldown period is also important. You may choose to use shift curves from your media mix model to inform this and/or set a cooldown period based on your typical customer journey. For example:
- Quick-converting products: 1-2 weeks
- Considered purchases: 2-4 weeks
- B2B/long sales cycles: 4-8+ weeks
4. Interpreting Your Results
When you analyze an incrementality experiment with GeoLift by Recast, you will receive the following:
- An incremental lift estimate: Additional revenue/conversions attributable to the test
- Statistical significance: Whether the lift is statistically meaningful
- Estimated ROI/CPA calculations: Based on your spend and lift
- ROI/CPA confidence intervals: The range of plausible test outcomes
When comparing these results to other methods you may have used in the past, there are a few things you can look for:
- Directional alignment: Do both analyses show positive/negative lift results?
- Magnitude differences: Are the effect sizes similar?
- Statistical significance: Does one method find significance where the other doesn’t?
After comparing lift test results across different methods, there are further next steps you can take:
- Build a Test Database: Create a shared database of all your incrementality tests analyzed with a consistent methodology
- Inform Future Tests: Use learnings from previous incrementality tests to inform better test design and execution in the future
- Calibrate Your MMM: If using Recast’s MMM platform, export your results to incorporate into your model (note: Recast MMM integrates with all lift testing platforms)
Remember, the goal isn’t to find the “right” answer but to get closer to measuring true incrementality with multiple measurement approaches. Reanalyzing past experiments with GeoLift by Recast adds another valuable perspective to your incrementality measurement toolkit.
