A/B testing with split_clients in Nginx - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Output of split_clients with 50/50 split

Given the following nginx configuration snippet using split_clients, what is the value of the variable $variant for the client with IP 192.168.1.10?

Nginx

split_clients "$remote_addr" $variant {
    50%     "A";
    *       "B";
}

# Assume $remote_addr is 192.168.1.10

A "B"

B "A"

C "50%"

D "192.168.1.10"

💡 Hint

The split_clients directive hashes the input and assigns variants based on percentage ranges.

The split_clients directive hashes the client IP and assigns the variant based on the defined percentages. Since 50% is assigned to "A" and the rest to "B", the client IP 192.168.1.10 falls into the first 50% bucket, so $variant is "A".

❓ Configuration

intermediate

Correct split_clients syntax for 30/70 split

Which of the following nginx split_clients configurations correctly splits traffic into 30% variant "X" and 70% variant "Y"?

A

split_clients "$cookie_user" $group {
    30% "X";
    *   "Y";
}

B

split_clients "$cookie_user" $group {
    70% "X";
    30% "Y";
}

C

split_clients "$cookie_user" $group {
    30% "X";
    70% "Y";
    *   "Z";
}

D

split_clients "$cookie_user" $group {
    30% "X";
    70% "Y";
}

💡 Hint

Percentages must add up to 100%, and the wildcard * covers the rest.

Option A correctly assigns 30% to "X" and the rest (70%) to "Y" using the wildcard *. Option A is invalid because percentages must be in descending order and the wildcard * must be last. Option A is correct syntax-wise but does not use wildcard; however, it is valid if percentages add to 100%. Option A has an extra wildcard causing invalid configuration.

❓ Troubleshoot

advanced

Why does split_clients always assign the same variant?

An nginx config uses split_clients "$http_user_agent" $variant but all users get assigned variant "A" even though the split is 50/50. What is the most likely cause?

Nginx

split_clients "$http_user_agent" $variant {
    50% "A";
    *    "B";
}

A The split_clients directive does not support string variables like $http_user_agent.

B The split_clients directive requires $remote_addr, not $http_user_agent.

C The percentages must be reversed to 50% "B" and * "A".

D The $http_user_agent variable is empty or identical for all users, causing the hash to always fall in the first bucket.

💡 Hint

Check what value is being hashed by split_clients.

If the variable used in split_clients is empty or the same for all users, the hash will always produce the same result, assigning the same variant. $http_user_agent can be identical or missing in some requests, causing this behavior.

🔀 Workflow

advanced

Order of directives for A/B testing with split_clients

What is the correct order of nginx directives to implement A/B testing using split_clients and setting a cookie for persistent variant assignment?

A 3,1,2,4

B 1,3,2,4

C 3,2,1,4

D 2,3,1,4

💡 Hint

You must extract the cookie first, then split, then set cookie if missing.

First, use map to extract the variant from the cookie. Then use split_clients to assign a variant if cookie is empty. Then set the cookie if it was missing. Finally, the server block contains these directives.

✅ Best Practice

expert

Best practice for consistent A/B testing with split_clients

Which approach ensures consistent user experience in nginx A/B testing using split_clients across multiple sessions?

A Use <code>split_clients</code> on <code>$remote_addr</code> only, without cookies.

B Use <code>split_clients</code> on <code>$http_user_agent</code> to vary variants by browser type.

C Use <code>split_clients</code> on a cookie value, and set the cookie if missing to persist variant.

D Use <code>split_clients</code> with random variable to assign variant on each request.

💡 Hint

Persistence across sessions requires storing variant in a cookie.

Using split_clients on a cookie value and setting the cookie if missing ensures the user gets the same variant on subsequent visits, providing a consistent experience. Using IP or user agent alone can cause inconsistent assignment.