Case Study
Lift Math — trade-promotion incrementality, scored against truth
Every incrementality vendor asserts accuracy. This is the version where you can check.
The question every CFO asks, and the answer nobody can audit
A specialty food brand commits real money to trade promotion and gets back a report that says the promotions worked. The report rests on one number — incremental lift, the sales the promotion caused rather than borrowed or bought — and that number rests on a counterfactual: what would have sold anyway. No store runs both versions of the week. There is no ground truth in retail scan data, which means every vendor’s accuracy claim is unauditable by construction. The industry’s benchmark studies put the share of trade promotions that fail to break even at roughly seven in ten; the reports brands actually receive rarely say anything so uncomfortable.
That gap — between what promotions return and what reports claim — is one of the most expensive unexamined numbers in a growing brand’s P&L.
What Lift Math does about it
Lift Math answers the question in three connected views, in the order a skeptical buyer actually asks it.
The verdict.An ROI scorecard over 131 promotion events: 45 of 129 estimable events lost money under Method 0 — the naive pre-period baseline, deliberately the most forgiving measure available. The portfolio clears 1.47× only because a thin tail of winners carries a middle that didn’t pay back. Toggle to the more defensible comparable-store baseline and the picture darkens: 1.35×, with 48 events under water.
The anatomy.Click any event and the waterfall shows where its promoted volume came from — gross promoted volume, the subsidized baseline that would have sold anyway, and the net incremental lift the estimator credits to the promotion — with the trade cost, manufacturer margin, and giveaway share alongside. This is the answer to the objection every trade lead raises about their own favorite event: show me why.
The proof. The accuracy view scores both estimators against known ground truth. Median error ~26% on incremental units for both methods. The more defensible method is moreupward-biased, not less. Error grows where the comparable-store match had to relax, spikes on shallow promotions and coupons, and breaks worst in winter — exactly where a pre-period baseline should break. The error is shown by regime, at full size, including where it is large.
No vendor shows you that page — which is why this one exists.
How the proof is possible
Ground truth exists here because the data is synthetic — the Cinderhaven universe, a ~$25M specialty food brand with 50 SKUs across six retailers, with a promotion-response layer whose true lift, pantry-load dip, and cannibalization are generated, locked, and then quarantined. The estimators are provably blind to it, and “provably” is meant literally: an AST gate runs in CI and rejects any estimation code that touches the truth artifact; the generator’s own coefficients are banned from the estimation path; and the git history shows both estimation methods specified, implemented, and tagged before any code in the repository first read truth. The blindness claim is scoped exactly there — to the code — and nowhere wider.
Synthetic data is not a compromise here. It is the only honest testbed twice over: the only world where truth is knowable at all, and the only world that can be published — real client promotion data can never be shown, by anyone. A vendor demonstrating accuracy on “real client data” is either breaching confidentiality or making it up.
What the findings mean
The one-line version: the true promo book is worse than any estimate shows. Both methods over-credit promotions — the calibrated truth has roughly two-thirds of events below break-even, while even the honest estimators find about a third. And the single most useful finding wasn’t scripted: the better method did not win. A demonstration engineered to flatter the sophisticated approach could never produce that result.
The tool also renders the trade-spend failure modes brands actually live with: phantom promotions that accrued cost and produced nothing, a deep-discount event that clears 1.4× — a vendor scorecard would call it a winner — while roughly half its trade dollars subsidized volume that was already moving. ROI alone hides the waste. The giveaway share exposes it.
What this looks like on your data
The engagement version runs the same estimators on your POS and promotion calendar, with the same honesty about what can and cannot be known. Nobody can score error on your data — that is the entire lesson — but the failure map transfers: the regimes where these methods degrade are observable features of your own calendar, and the deliverable tells you which of your events sit in them, which promotions lost money under the most forgiving read available, and what each trade dollar actually bought.