Incrementality Engine
Separate genuinely new demand from orders you would have won anyway, and get a true cost per incremental order.
A VP of Growth
at a large on-demand food-delivery & quick-commerce platform
India
Forsyt Labs
A Revenue Community perk
They were running a dozen growth bets in parallel, express delivery, instant grocery, dine-out, new-city launches, cashback pushes, and every one of them looked like a winner in its own dashboard. The problem was obvious to her and invisible to the numbers: a huge share of the orders each initiative claimed would have happened anyway through another channel. Cost per order looked artificially cheap, and money was being poured into initiatives that were mostly cannibalising each other. She wanted one honest number per initiative: net-new orders, cannibalization stripped out, and what an incremental order actually costs.
Net incremental orders per day and true cost per incremental order up top, then a per-initiative breakdown of gross vs. substituted vs. net-new, with cannibalization and true CPO for each bet.
The three causal methods side by side with their lift and confidence intervals. When they converge within the tolerance band the read is trusted; cannibalization is broken out by city tier below.
Overview
Incrementality Engine measures the true, net-new impact of each growth initiative instead of the inflated number on its own dashboard. It runs three complementary causal methods continuously, encouragement design, a matched customer-cohort panel, and capacity-neutral switchbacks, and cross-validates them. When they converge, you get a trustworthy read: net incremental orders per day per initiative, cannibalization removed, and a true cost per incremental order. Everything upstream of a thin data connector is platform-agnostic.
Why we built this
When several growth initiatives run at once, each one takes credit for orders that would have happened anyway through another channel. Every dashboard looks like a success, cost per order looks artificially low, and investment decisions get made on inflated numbers. Incrementality Engine strips out substitution and cannibalization so you fund what actually grows the business, not what merely reshuffles existing demand. No single method is trusted alone; the three are run together precisely because each corrects the others’ blind spot.
How it works
- 01Encouragement design
Randomly vary how prominently an initiative is surfaced (sort order, banner, nudge) to a slice of eligible users. The feature stays live for everyone, only discovery is randomized, so there is no supply shock or operational disruption.
- 02Matched customer-cohort panel
Build a rolling behavioural history per customer and match adopters to statistically similar non-adopters on prior frequency, basket size, geo, and cuisine mix (propensity / double ML). This is also where cannibalization is measured directly.
- 03Capacity-neutral switchbacks
Short randomized on/off discovery windows within a single market, refreshed on a rolling schedule, give a continuous daily read that never goes stale.
- 04Triangulate & report
Cross-validate the three reads. When they converge within a tight tolerance band, the number is trusted and published, net incremental orders per day and true cost per incremental order, to a stakeholder dashboard and an API.
Capabilities
- Net incremental orders per day, per initiative, with cannibalization removed.
- True cost per incremental order, next to the (usually far lower) reported cost per order.
- Three causal methods run continuously; convergence within a tolerance band signals trust.
- Cannibalization measured directly from a matched cohort panel, broken out by cohort and city tier.
- Encouragement + switchback designs randomize discovery only, no supply shock, no ops disruption.
- Decoupled architecture: a thin connector maps your schema to a generic one, so it ports to any platform.
- Stakeholder dashboard plus an API so other tools consume the same honest numbers.