Incrementality Testing (Holdout Tests)
Incrementality Testing is a causal measurement method that establishes the true incremental effect of a marketing activity by exposing a test group – a region or user segment, for example – to the advertising while a comparable control or holdout group sees none. The most common form is the geo holdout test, in which geographic regions are split into test and control markets so that the difference in revenue can be attributed statistically. Unlike attribution models, which reconstruct click paths at user level, Incrementality Testing measures in aggregate and independently of cookies or tracking signal loss.
In practice
For agencies, Incrementality Testing is the method of choice when clients doubt the value of click attribution, for instance after the iOS tracking restrictions, or want to check large campaigns across TV, social and programmatic for real revenue effect. As a rule of thumb, a typical geo test runs for 4–8 weeks, needs cleanly comparable test and control regions, and produces figures such as incremental ROAS (iROAS); the runtime actually required follows from the minimum detectable effect you are aiming for, the variance of revenue and the number of comparable markets. Unlike A/B tests, which compare individual page variants at user level, Incrementality Testing examines whole channels or campaigns at market level. Platform-native holdout tests such as Meta Conversion Lift or Google Ads Geo Experiments reduce the setup effort; Google Ads Conversion Lift, however, is not available for every account and is enabled only through your Google contact – so for smaller companies without account management it is no easy entry point. For robust cross-channel conclusions, specialised tools such as GeoLift are worth considering. Results should be reconciled with attribution data and MMM rather than read in isolation.