Case Study
Lift Math: trade-promotion incrementality, scored against truth
Every incrementality vendor asserts accuracy. This is the version where you can check.
Figures on this page are as of package v0.6.1; the live tool computes its own from artifacts on every build.
The question every CFO asks, and the answer nobody can audit
A specialty food brand commits real money to trade promotion and gets back a report that says the promotions worked. The report rests on one number, incremental lift: the sales the promotion caused rather than borrowed or bought. And that number rests on a counterfactual, what would have sold anyway. No store runs both versions of the week. There is no ground truth in retail scan data, which means every vendor’s accuracy claim is unauditable by construction. McKinsey puts the share of US trade promotions that fail to break even at 72%; the reports brands actually receive rarely say anything so uncomfortable.
That gap, between what promotions return and what reports claim, is one of the most expensive unexamined numbers in a growing brand’s P&L. It is also where the money leaks after the commitment: the trade spend and deduction recovery work picks up that half of the story.
What Lift Math does about it
Lift Math answers the question in three connected views, in the order a skeptical buyer actually asks it.
The verdict. An ROI scorecard over 5,897 promotion events: 2,170 of 5,735 estimable events lost money under Method 0, the naive pre-period baseline, deliberately the most forgiving measure available. The portfolio clears 1.38x only because a thin slice of winners carries it: the top tenth of events produce 58% of the net margin. Toggle to the more defensible comparable-store baseline and the picture darkens: 1.18x, with 2,470 events under water.
The anatomy. Click any event and the waterfall shows where its promoted volume came from: gross promoted volume, the subsidized baseline that would have sold anyway, and the net incremental lift the estimator credits to the promotion, with the trade cost, manufacturer margin, and giveaway share alongside. This is the answer to the objection every trade lead raises about their own favorite event: show me why.
The proof. The accuracy view scores both estimators against known ground truth. Median error is 33% on incremental units for the naive method and 42% for the comparable-store method: the more defensible method is both more wrong and more upward-biased (+27% versus +18%). Error spikes on shallow promotions and coupons, and breaks worst in fall and winter, exactly where a pre-period baseline should break. It is shown by regime, at full size, including where it is large.
No vendor shows you that page, which is why this one exists.
How the proof is possible
Ground truth exists here because the data is synthetic: the Cinderhaven universe, a $25M specialty food brand with 50 SKUs across six retailers running a calendar of 5,897 promotion events, about a third of volume on promotion, with a promotion-response layer whose true lift is generated, locked, and quarantined. The estimators are provably blind to it, and provably is meant literally: a code gate rejects any estimation code that touches the truth artifact, and the git history shows both methods frozen and tagged before anything in the repository first read truth.
Synthetic data is not a compromise here. It is the only honest testbed twice over: the only world where truth is knowable at all, and the only world that can be published. Real client promotion data can never be shown, by anyone. A vendor demonstrating accuracy on real client data is either breaching confidentiality or making it up.
What the findings mean
The one-line version: the true promo book is worse than any estimate shows. The truth is calibrated to the industry’s own seven-in-ten failure benchmark, while even the honest estimators find roughly four in ten to nearly half. Both methods over-credit promotions, so every number a brand has ever been shown was, if anything, flattering. And the single most useful finding was not scripted: the better method did not win. The more defensible comparable-store baseline over-credits by more, not less. A demonstration engineered to flatter the sophisticated approach could never produce that result.
The tool also renders the failure modes trade teams actually live with: phantom promotions that accrued cost and produced nothing, and a deep-discount event that clears 1.14x, a number a vendor scorecard would call a winner, while a 60% giveaway share shows most of its trade dollars subsidizing volume that was already moving. ROI alone hides the waste. The giveaway share exposes it. The promotion process itself breaks the same way: three data owners, no reconciliation, spend nobody measures.
What this looks like on your data
The engagement version runs the same estimators on your POS and promotion calendar, with the same honesty about what can and cannot be known. Nobody can score error on your data; that is the entire lesson. But the failure map transfers: the regimes where these methods degrade are observable features of your own calendar, and the deliverable tells you which of your events sit in them, which promotions lost money under the most forgiving read available, and what each trade dollar actually bought. Fixed fee, defined timeline, deliverables that stand on their own.
Data is synthetic; methodology and deliverables are real.
Next step — Trade spend nobody can account for
Find out what it is costing you. Free, no call.
The worked example above is synthetic so the method can be shown in full. The offers below run it on your data — the scan is free, and the Snapshot credits in full toward the audit.
Private, expiring upload — never email. Mutual NDA before anything moves. Files destroyed within 30 days of delivery, with a certificate. Methods published, tools open source.
Start in writing.
A few minutes by form, no call. Tell me how you measure promotions today, and I will tell you what the most forgiving honest read of your promo book would show. No deck, no obligation.