A/B testing with split_clients in Nginx - Step-by-Step Execution

Process Flow - A/B testing with split_clients

Client Request Arrives

↓

split_clients Directive

↓

Hash Client ID or IP

↓

Assign Client to Variant A or B

↓

Serve Variant Content

↓

Log or Track Variant Served

The client request is processed by split_clients which hashes client info to assign them to variant groups for A/B testing.

Execution Sample

Nginx

split_clients "$remote_addr" $variant {
  50%     "A";
  *       "B";
}

if ($variant = "A") {
  return 200 "Variant A served";
}
return 200 "Variant B served";

This config splits clients by IP address into two groups: 50% get variant A, others get variant B, and returns a message accordingly.

Process Table

Step	Client IP	Hash Result	Assigned Variant	Action	Output
1	192.168.1.10	hash=0.45	A	Serve Variant A	Variant A served
2	192.168.1.11	hash=0.75	B	Serve Variant B	Variant B served
3	10.0.0.5	hash=0.10	A	Serve Variant A	Variant A served
4	10.0.0.6	hash=0.90	B	Serve Variant B	Variant B served
5	172.16.0.1	hash=0.50	A	Serve Variant A	Variant A served
6	172.16.0.2	hash=0.99	B	Serve Variant B	Variant B served
7	End of requests	-	-	-	No more clients

💡 All client requests processed and assigned variants based on hash thresholds.

Status Tracker

Variable	Start	After 1	After 2	After 3	After 4	After 5	After 6	Final
$remote_addr	-	192.168.1.10	192.168.1.11	10.0.0.5	10.0.0.6	172.16.0.1	172.16.0.2	-
$variant	-	A	B	A	B	A	B	-

Key Moments - 3 Insights

Why does a client with hash exactly 0.50 get assigned to variant A?

Because the split_clients directive assigns 50% to variant A using '50%', which includes hash values up to but not including 0.50. However, in nginx split_clients, the percentage is inclusive of the boundary, so hash=0.50 maps to A as shown in execution_table row 5.

What happens if a client's hash is greater than 0.50?

They are assigned to variant B as the '*' catch-all rule covers all remaining hash values above 0.50, as seen in rows 2,4,6.

Is the variant assignment consistent for the same client IP?

Yes, because the hash is based on the client IP ($remote_addr), so the same IP always hashes to the same variant, ensuring consistent experience.

Visual Quiz - 3 Questions

Look at the execution_table, what variant is assigned to client IP 10.0.0.5?

A Variant B

B No variant assigned

C Variant A

D Depends on time

💡 Hint

Check row 3 in execution_table where client IP 10.0.0.5 is assigned variant A.

At which step does the hash value exceed 0.50, causing assignment to variant B?

A Step 1

B Step 2

C Step 3

D Step 5

💡 Hint

Look at execution_table row 2 where hash=0.75 > 0.50, assigned to B.

If the split_clients directive changed to 30% for variant A, how would the assignment change for hash=0.45?

A Assigned to variant B

B Assigned to variant A

C No assignment

D Random assignment

💡 Hint

Since 0.45 > 0.30, it falls outside the 30% threshold, so assigned to variant B.

Concept Snapshot

split_clients "$variable" $target {
  percentage% "variant1";
  * "variant2";
}

- Splits clients by hashing $variable (e.g., IP)
- Assigns variants based on percentage thresholds
- Ensures consistent variant per client
- Used for A/B testing in nginx

Full Transcript

This visual execution shows how nginx's split_clients directive assigns clients to A/B test variants. Each client IP is hashed to a number between 0 and 1. If the hash falls within the first percentage (e.g., 50%), the client is assigned variant A; otherwise, variant B. The execution table traces multiple client IPs, their hash values, assigned variants, and the response served. Variables $remote_addr and $variant track client IP and assigned variant respectively. Key moments clarify how boundary hash values are assigned and consistency of assignment. The quiz tests understanding of variant assignment based on hash thresholds. The snapshot summarizes the syntax and behavior of split_clients for quick reference.

Practice

1. What is the main purpose of the split_clients directive in nginx for A/B testing?

easy

A. To redirect all traffic to a single URL

B. To cache static files for faster loading

C. To compress responses before sending to clients

D. To divide visitors into groups based on a variable for testing different versions

Solution

1. Step 1: Understand the role of split_clients

   The split_clients directive is designed to split traffic into groups based on a variable like IP or cookie.

2. Step 2: Connect to A/B testing purpose

   This splitting allows serving different site versions to different groups, which is the core of A/B testing.

3. Final Answer:

   To divide visitors into groups based on a variable for testing different versions -> Option D

4. Quick Check:

   split_clients divides visitors for testing [OK]

Hint: Remember: split_clients splits visitors into groups [OK]

Common Mistakes:

• Confusing split_clients with caching directives
• Thinking it redirects all traffic to one URL
• Assuming it compresses responses

2. Which of the following is the correct syntax to use split_clients in nginx to split traffic by cookie $cookie_user into two groups: 50% 'A' and 50% 'B'?

easy

A. split_clients $cookie_user $variant { 50% A; 50% B; }

B. split_clients $cookie_user { 50% A; 50% B; }

C. split_clients $cookie_user $variant { 50 A; 50 B; }

D. split_clients $cookie_user $variant { 50% "A"; 50% "B"; }

Solution

1. Step 1: Review split_clients syntax

   The correct syntax requires specifying the source variable, the destination variable, and the percentage groups with semicolons.

2. Step 2: Check each option

   split_clients $cookie_user $variant { 50% A; 50% B; } correctly uses split_clients $cookie_user $variant { 50% A; 50% B; }. split_clients $cookie_user { 50% A; 50% B; } misses the destination variable. split_clients $cookie_user $variant { 50 A; 50 B; } lacks the percent sign. split_clients $cookie_user $variant { 50% "A"; 50% "B"; } incorrectly quotes the group names.

3. Final Answer:

   split_clients $cookie_user $variant { 50% A; 50% B; } -> Option A

4. Quick Check:

   Syntax requires source, destination, and percent groups [OK]

Hint: Syntax: split_clients source_var dest_var { percent group; } [OK]

Common Mistakes:

• Omitting the destination variable
• Forgetting percent signs (%)
• Using quotes around group names

3. Given this nginx config snippet:

split_clients $remote_addr $group {
  30% A;
  70% B;
}

server {
  location / {
    if ($group = A) {
      return 200 'Group A content';
    }
    return 200 'Group B content';
  }
}

What response do 70% of visitors receive?

medium

A. Group A content

B. Error: invalid syntax

C. Group B content

D. No response, connection closed

Solution

1. Step 1: Understand split_clients distribution

   The split_clients directive splits visitors by IP into 30% group A and 70% group B.

2. Step 2: Identify the majority group

   The configuration assigns 70% of visitors to group B based on the hash distribution, so they receive 'Group B content'.

3. Final Answer:

   Group B content -> Option C

4. Quick Check:

   70% of visitors get Group B content [OK]

Hint: Larger split_clients percentage gets more visitors [OK]

Common Mistakes:

• Assuming visitors always get Group A content
• Thinking syntax error occurs
• Misunderstanding the traffic split percentages

4. You wrote this nginx config:

split_clients $cookie_user $group {
  50% A
  50% B
}

But nginx fails to start. What is the error?

medium

A. Percentages must add up to 1000%

B. Missing semicolons after percentage groups

C. Destination variable $group is not declared

D. Using $cookie_user instead of $remote_addr

Solution

1. Step 1: Check syntax for split_clients block

   Each percentage group line must end with a semicolon (;). The config misses semicolons after '50% A' and '50% B'.

2. Step 2: Validate other options

   Using $cookie_user is valid. $group is declared as destination variable. Percentages add to 100%, not 1000%.

3. Final Answer:

   Missing semicolons after percentage groups -> Option B

4. Quick Check:

   Each group line ends with ; [OK]

Hint: Always end split_clients group lines with semicolons [OK]

Common Mistakes:

• Forgetting semicolons after group percentages
• Thinking variable names must be IP only
• Misreading percentage sums

5. You want to run an A/B test with three groups: 20% 'A', 30% 'B', and 50% 'C'. You also want to ensure users stay in the same group on repeat visits using a cookie $cookie_user. Which nginx config snippet correctly implements this?

hard

A. split_clients $cookie_user $group {
20% A;
30% B;
50% C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

B. split_clients $remote_addr $group {
20% A;
30% B;
50% C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

C. split_clients $cookie_user $group {
20 A;
30 B;
50 C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

D. split_clients $cookie_user $group {
20% A
30% B
50% C
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

Solution

1. Step 1: Confirm correct split_clients syntax and variable

   split_clients $cookie_user $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } uses $cookie_user to keep users in the same group on repeat visits and correctly uses percentages with semicolons.

2. Step 2: Check other options for errors

   split_clients $remote_addr $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } uses $remote_addr which may change and not guarantee consistent grouping. split_clients $cookie_user $group { 20 A; 30 B; 50 C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } misses percent signs. split_clients $cookie_user $group { 20% A 30% B 50% C } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } misses semicolons after group lines.

3. Final Answer:

   split_clients $cookie_user $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } -> Option A

4. Quick Check:

   Use cookie variable and proper syntax for consistent groups [OK]

Hint: Use cookie variable and semicolons for stable multi-group split [OK]

Common Mistakes:

• Using IP instead of cookie for consistent user grouping
• Omitting percent signs or semicolons
• Misordering or missing group percentages