A/B testing with split_clients in Nginx - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Output of split_clients with 50/50 split

Given the following nginx configuration snippet using split_clients, what is the value of the variable $variant for the client with IP 192.168.1.10?

Nginx

split_clients "$remote_addr" $variant {
    50%     "A";
    *       "B";
}

# Assume $remote_addr is 192.168.1.10

A "B"

B "A"

C "50%"

D "192.168.1.10"

💡 Hint

The split_clients directive hashes the input and assigns variants based on percentage ranges.

The split_clients directive hashes the client IP and assigns the variant based on the defined percentages. Since 50% is assigned to "A" and the rest to "B", the client IP 192.168.1.10 falls into the first 50% bucket, so $variant is "A".

❓ Configuration

intermediate

Correct split_clients syntax for 30/70 split

Which of the following nginx split_clients configurations correctly splits traffic into 30% variant "X" and 70% variant "Y"?

A

split_clients "$cookie_user" $group {
    30% "X";
    *   "Y";
}

B

split_clients "$cookie_user" $group {
    70% "X";
    30% "Y";
}

C

split_clients "$cookie_user" $group {
    30% "X";
    70% "Y";
    *   "Z";
}

D

split_clients "$cookie_user" $group {
    30% "X";
    70% "Y";
}

💡 Hint

Percentages must add up to 100%, and the wildcard * covers the rest.

Option A correctly assigns 30% to "X" and the rest (70%) to "Y" using the wildcard *. Option A is invalid because percentages must be in descending order and the wildcard * must be last. Option A is correct syntax-wise but does not use wildcard; however, it is valid if percentages add to 100%. Option A has an extra wildcard causing invalid configuration.

❓ Troubleshoot

advanced

Why does split_clients always assign the same variant?

An nginx config uses split_clients "$http_user_agent" $variant but all users get assigned variant "A" even though the split is 50/50. What is the most likely cause?

Nginx

split_clients "$http_user_agent" $variant {
    50% "A";
    *    "B";
}

A The split_clients directive does not support string variables like $http_user_agent.

B The split_clients directive requires $remote_addr, not $http_user_agent.

C The percentages must be reversed to 50% "B" and * "A".

D The $http_user_agent variable is empty or identical for all users, causing the hash to always fall in the first bucket.

💡 Hint

Check what value is being hashed by split_clients.

If the variable used in split_clients is empty or the same for all users, the hash will always produce the same result, assigning the same variant. $http_user_agent can be identical or missing in some requests, causing this behavior.

🔀 Workflow

advanced

Order of directives for A/B testing with split_clients

What is the correct order of nginx directives to implement A/B testing using split_clients and setting a cookie for persistent variant assignment?

A 3,1,2,4

B 1,3,2,4

C 3,2,1,4

D 2,3,1,4

💡 Hint

You must extract the cookie first, then split, then set cookie if missing.

First, use map to extract the variant from the cookie. Then use split_clients to assign a variant if cookie is empty. Then set the cookie if it was missing. Finally, the server block contains these directives.

✅ Best Practice

expert

Best practice for consistent A/B testing with split_clients

Which approach ensures consistent user experience in nginx A/B testing using split_clients across multiple sessions?

A Use <code>split_clients</code> on <code>$remote_addr</code> only, without cookies.

B Use <code>split_clients</code> on <code>$http_user_agent</code> to vary variants by browser type.

C Use <code>split_clients</code> on a cookie value, and set the cookie if missing to persist variant.

D Use <code>split_clients</code> with random variable to assign variant on each request.

💡 Hint

Persistence across sessions requires storing variant in a cookie.

Using split_clients on a cookie value and setting the cookie if missing ensures the user gets the same variant on subsequent visits, providing a consistent experience. Using IP or user agent alone can cause inconsistent assignment.

Practice

1. What is the main purpose of the split_clients directive in nginx for A/B testing?

easy

A. To redirect all traffic to a single URL

B. To cache static files for faster loading

C. To compress responses before sending to clients

D. To divide visitors into groups based on a variable for testing different versions

Solution

1. Step 1: Understand the role of split_clients

   The split_clients directive is designed to split traffic into groups based on a variable like IP or cookie.

2. Step 2: Connect to A/B testing purpose

   This splitting allows serving different site versions to different groups, which is the core of A/B testing.

3. Final Answer:

   To divide visitors into groups based on a variable for testing different versions -> Option D

4. Quick Check:

   split_clients divides visitors for testing [OK]

Hint: Remember: split_clients splits visitors into groups [OK]

Common Mistakes:

• Confusing split_clients with caching directives
• Thinking it redirects all traffic to one URL
• Assuming it compresses responses

2. Which of the following is the correct syntax to use split_clients in nginx to split traffic by cookie $cookie_user into two groups: 50% 'A' and 50% 'B'?

easy

A. split_clients $cookie_user $variant { 50% A; 50% B; }

B. split_clients $cookie_user { 50% A; 50% B; }

C. split_clients $cookie_user $variant { 50 A; 50 B; }

D. split_clients $cookie_user $variant { 50% "A"; 50% "B"; }

Solution

1. Step 1: Review split_clients syntax

   The correct syntax requires specifying the source variable, the destination variable, and the percentage groups with semicolons.

2. Step 2: Check each option

   split_clients $cookie_user $variant { 50% A; 50% B; } correctly uses split_clients $cookie_user $variant { 50% A; 50% B; }. split_clients $cookie_user { 50% A; 50% B; } misses the destination variable. split_clients $cookie_user $variant { 50 A; 50 B; } lacks the percent sign. split_clients $cookie_user $variant { 50% "A"; 50% "B"; } incorrectly quotes the group names.

3. Final Answer:

   split_clients $cookie_user $variant { 50% A; 50% B; } -> Option A

4. Quick Check:

   Syntax requires source, destination, and percent groups [OK]

Hint: Syntax: split_clients source_var dest_var { percent group; } [OK]

Common Mistakes:

• Omitting the destination variable
• Forgetting percent signs (%)
• Using quotes around group names

3. Given this nginx config snippet:

split_clients $remote_addr $group {
  30% A;
  70% B;
}

server {
  location / {
    if ($group = A) {
      return 200 'Group A content';
    }
    return 200 'Group B content';
  }
}

What response do 70% of visitors receive?

medium

A. Group A content

B. Error: invalid syntax

C. Group B content

D. No response, connection closed

Solution

1. Step 1: Understand split_clients distribution

   The split_clients directive splits visitors by IP into 30% group A and 70% group B.

2. Step 2: Identify the majority group

   The configuration assigns 70% of visitors to group B based on the hash distribution, so they receive 'Group B content'.

3. Final Answer:

   Group B content -> Option C

4. Quick Check:

   70% of visitors get Group B content [OK]

Hint: Larger split_clients percentage gets more visitors [OK]

Common Mistakes:

• Assuming visitors always get Group A content
• Thinking syntax error occurs
• Misunderstanding the traffic split percentages

4. You wrote this nginx config:

split_clients $cookie_user $group {
  50% A
  50% B
}

But nginx fails to start. What is the error?

medium

A. Percentages must add up to 1000%

B. Missing semicolons after percentage groups

C. Destination variable $group is not declared

D. Using $cookie_user instead of $remote_addr

Solution

1. Step 1: Check syntax for split_clients block

   Each percentage group line must end with a semicolon (;). The config misses semicolons after '50% A' and '50% B'.

2. Step 2: Validate other options

   Using $cookie_user is valid. $group is declared as destination variable. Percentages add to 100%, not 1000%.

3. Final Answer:

   Missing semicolons after percentage groups -> Option B

4. Quick Check:

   Each group line ends with ; [OK]

Hint: Always end split_clients group lines with semicolons [OK]

Common Mistakes:

• Forgetting semicolons after group percentages
• Thinking variable names must be IP only
• Misreading percentage sums

5. You want to run an A/B test with three groups: 20% 'A', 30% 'B', and 50% 'C'. You also want to ensure users stay in the same group on repeat visits using a cookie $cookie_user. Which nginx config snippet correctly implements this?

hard

A. split_clients $cookie_user $group {
20% A;
30% B;
50% C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

B. split_clients $remote_addr $group {
20% A;
30% B;
50% C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

C. split_clients $cookie_user $group {
20 A;
30 B;
50 C;
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

D. split_clients $cookie_user $group {
20% A
30% B
50% C
}

server {
location / {
if ($group = A) { return 200 'A version'; }
if ($group = B) { return 200 'B version'; }
return 200 'C version';
}
}

Solution

1. Step 1: Confirm correct split_clients syntax and variable

   split_clients $cookie_user $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } uses $cookie_user to keep users in the same group on repeat visits and correctly uses percentages with semicolons.

2. Step 2: Check other options for errors

   split_clients $remote_addr $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } uses $remote_addr which may change and not guarantee consistent grouping. split_clients $cookie_user $group { 20 A; 30 B; 50 C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } misses percent signs. split_clients $cookie_user $group { 20% A 30% B 50% C } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } misses semicolons after group lines.

3. Final Answer:

   split_clients $cookie_user $group { 20% A; 30% B; 50% C; } server { location / { if ($group = A) { return 200 'A version'; } if ($group = B) { return 200 'B version'; } return 200 'C version'; } } -> Option A

4. Quick Check:

   Use cookie variable and proper syntax for consistent groups [OK]

Hint: Use cookie variable and semicolons for stable multi-group split [OK]

Common Mistakes:

• Using IP instead of cookie for consistent user grouping
• Omitting percent signs or semicolons
• Misordering or missing group percentages