A/B testing ad variations in Digital Marketing - Step-by-Step Execution

Concept Flow - A/B testing ad variations

Create two ad versions: A and B

↓

Show ads randomly to users

↓

Collect user responses (clicks, conversions)

↓

Compare performance metrics

↓

Choose best ad

↓

Implement winning ad

This flow shows how two ad versions are created, shown randomly, measured, compared, and the best one is chosen.

Execution Sample

Digital Marketing

Ad_A = 'Discount 10%'
Ad_B = 'Free shipping'
Show ads randomly
Collect clicks
Calculate CTR
Compare CTR
Choose winner

This example shows two ads tested by measuring click-through rates to pick the better one.

Analysis Table

Step	Action	Ad Shown	Clicks Collected	CTR Calculated	Decision
1	Show Ad A to 100 users	A	20	20%	Continue
2	Show Ad B to 100 users	B	15	15%	Continue
3	Compare CTR	-	-	-	Ad A has higher CTR
4	Choose winner	-	-	-	Select Ad A for campaign
5	End test	-	-	-	Testing stops

💡 Test ends after comparing CTR and selecting the better performing ad.

State Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
Clicks_A	0	20	20	20	20
Clicks_B	0	0	15	15	15
CTR_A	0%	20%	20%	20%	20%
CTR_B	0%	0%	15%	15%	15%
Winner	None	None	None	Ad A	Ad A

Key Insights - 3 Insights

Why do we show ads randomly to users?

Showing ads randomly ensures fair comparison by avoiding bias, as seen in steps 1 and 2 of the execution_table.

What does CTR mean and why is it important?

CTR (Click-Through Rate) measures how many users clicked the ad out of those who saw it; it's the key metric to decide the better ad (step 3).

Why do we stop the test after choosing the winner?

Once the better ad is identified (step 4), continuing the test wastes resources; so we stop and implement the winning ad.

Visual Quiz - 3 Questions

Look at the execution_table, what is the CTR of Ad B after step 2?

A 15%

B 20%

C 0%

D 35%

💡 Hint

Check the 'CTR Calculated' column in row for step 2.

At which step does the test decide which ad is better?

A Step 1

B Step 3

C Step 5

D Step 2

💡 Hint

Look for the step where 'Compare CTR' and 'Decision' columns show the winner.

If Ad B had 25 clicks instead of 15 at step 2, what would happen?

A Ad A would still win

B Ad B would win

C Test would continue without decision

D Both ads would be rejected

💡 Hint

Compare clicks and CTR values in variable_tracker for both ads.

Concept Snapshot

A/B testing compares two ad versions by showing them randomly to users.
Measure user responses like clicks to calculate CTR.
Compare CTRs to find the better ad.
Choose and implement the winning ad.
This helps improve ad effectiveness based on real user data.

Full Transcript

A/B testing ad variations involves creating two different ads and showing them randomly to users. We collect data on how many users click each ad, then calculate the click-through rate (CTR) for each. By comparing these CTRs, we find which ad performs better. Once identified, we select the winning ad and stop the test. This process ensures we use the most effective ad to reach users.