It is a paradox of statistics that experiments with small sample sizes are more likely to find larger effects of an intervention, when they find anything at all. This happens because smaller samples produce noisier estimates. For a result to clear the significance bar in a noisy environment, the observed […]
The Winner’s Curse: Why Underpowered Experiments Are Likely to Be Extra Wrong Read More »
Data can show you what happened and be used to project into the future with some degree of confidence, but it can never tell you why things are the way they are. No matter how massive your dataset, how sophisticated your pipeline, or how tight your confidence intervals, raw data
Don’t Be the Turkey: An Intro to Causal Thinking Read More »
How to use this article: While preparing for your work day, you start to make your morning coffee and realize you’re completely out of grounds. Worst day ever. Now you need to figure out the fastest way to get from your home to the grocery store and back again so
A redditor recently asked us how to break into causal inference when their current company doesn’t offer opportunities to work on it. This is a common roadblock for data professionals trying to build expertise in a new area, so we’re sharing our response publicly for anyone facing the same challenge.
You Don’t Need Permission to Learn Causal Inference Read More »
At Recast we spend a lot of time thinking about how to evaluate whether or not a model is “good”. People new to Recast are often surprised when we tell them that while we care a lot about checking to see if a model is good, we generally don’t use
Goodness of Fit: Moving Beyond R-Squared and MAPE in Complex Models Read More »
Time series data is special because the order in which observations occur is meaningful. For example, looking at sunglasses sales at Sun-Glasses-For-All (an imaginary brand), we can see that knowing yesterday’s sales gives you information about today’s sales. And sales this time last week gives you information about today’s sales
Gaussian Processes for Time Series Read More »
Garbage in, garbage out. If a media mix model has any chance of working, the model has to be fitted on high-quality, accurate data. Keep in mind: marketing mix models are holistic models operating at an aggregate level, not at an individual channel level. What that means is that if
Automated Data Quality Checks at Recast Read More »
Many businesses face tough challenges when it comes to measuring marketing incrementality because the nature of their product implies conversions that happen at multiple stages. Marketing activity drives new users or customers to the first stage, and then some proportion of those customers “convert” into a subsequent stage. This may
Product Update: Multi-Stage Models Read More »
In today’s rapidly evolving retail environment, it can be a real challenge for omnichannel brands to measure the incremental impact of their marketing efforts. The modern customer journey spans multiple touchpoints — a consumer might browse for a product category online, see a TV ad, get driven to an owned/D2C
How Omnichannel Brands Leverage Marketing Mix Modeling (MMM) Read More »
“I ran an incrementality test showing positive lift but didn’t see the results in the bottom line.” Sound familiar? I think we’ve all had this experience: we set up a nice, clean incrementality test to measure the effectiveness of a specific paid media channel. We get the results back: 5%
The Hidden Pitfall in Marketing Experiments: Understanding Uncertainty Read More »
