Incrementality testing in Digital Marketing - Step-by-Step Execution

Concept Flow - Incrementality testing

Start with Audience

↓

Split into Test & Control Groups

↓

Apply Marketing Campaign to Test Group Only

↓

Measure Outcomes in Both Groups

↓

Compare Results: Test vs Control

↓

Calculate Incremental Impact

↓

Decide on Campaign Effectiveness

Incrementality testing splits an audience into two groups, applies a campaign to one, then compares results to find the true impact.

Execution Sample

Digital Marketing

Audience = 1000 people
Split: Test=500, Control=500
Apply campaign to Test only
Measure sales Test=150, Control=100
Increment = 150 - 100 = 50

This example shows splitting an audience, running a campaign on one group, and calculating the incremental sales caused by the campaign.

Analysis Table

Step	Action	Test Group Outcome	Control Group Outcome	Incremental Impact Calculation
1	Split audience into Test and Control	500 people	500 people	N/A
2	Apply campaign to Test group only	Campaign active	No campaign	N/A
3	Measure sales in Test group	150 sales	N/A	N/A
4	Measure sales in Control group	N/A	100 sales	N/A
5	Calculate incremental impact	N/A	N/A	150 - 100 = 50 sales
6	Interpret result	N/A	N/A	Campaign caused 50 extra sales

💡 Test and Control groups measured; incremental impact calculated as difference in outcomes.

State Tracker

Variable	Start	After Split	After Campaign	After Measurement	Final
Audience Size	1000	Test=500, Control=500	Same	Same	Same
Campaign Applied	No	No	Test=Yes, Control=No	Same	Same
Sales	0	0	0	Test=150, Control=100	Increment=50

Key Insights - 3 Insights

Why do we need a control group that does not see the campaign?

The control group shows what would have happened without the campaign, so comparing it with the test group reveals the true incremental effect (see execution_table rows 4 and 5).

Can we just look at sales in the test group to know the campaign's success?

No, because sales might have increased anyway. The control group's sales provide a baseline to subtract from test group sales (see execution_table rows 3 and 4).

What if the test and control groups are not similar?

Differences in groups can bias results. Proper random splitting ensures groups are alike, making the incremental impact valid (see variable_tracker 'After Split').

Visual Quiz - 3 Questions

Looking at the execution_table, what is the sales number for the control group at step 4?

A 150 sales

B 50 sales

C 100 sales

D 0 sales

💡 Hint

Check the 'Control Group Outcome' column at step 4 in the execution_table.

At which step in the execution_table is the incremental impact calculated?

A Step 5

B Step 3

C Step 2

D Step 6

💡 Hint

Look for the step where the difference between test and control sales is computed.

If the control group sales were 120 instead of 100, how would the incremental impact change?

A It would increase to 70 sales

B It would decrease to 30 sales

C It would stay 50 sales

D It would become negative

💡 Hint

Use the formula Increment = Test sales - Control sales with new control sales from variable_tracker.

Concept Snapshot

Incrementality testing splits an audience into test and control groups.
Only the test group sees the campaign.
Measure outcomes in both groups.
Subtract control results from test results.
The difference shows the campaign's true impact.

Full Transcript

Incrementality testing is a method in digital marketing to find out if a campaign truly causes more sales or actions. First, you split your audience into two groups: test and control. The test group gets the campaign, the control group does not. Then you measure the results in both groups. By subtracting the control group's results from the test group's results, you find the incremental impact caused by the campaign. This helps marketers understand if their efforts are effective or if results would have happened anyway.