A/B testing with Ingress in Kubernetes - Commands & Configuration

Introduction

A/B testing helps you compare two versions of an app by sending some users to one version and others to a different version. Using Kubernetes Ingress, you can control traffic routing to different app versions easily without changing your app code.

When you want to test a new feature on a small group of users before full release

When you want to compare performance between two app versions running in parallel

When you want to gradually shift traffic from an old version to a new version

When you want to route users based on specific rules like cookies or headers

When you want to run experiments to improve user experience without downtime

Config File - ingress-ab-testing.yaml

ingress-ab-testing.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: stable-service
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary-service
            port:
              number: 80

This Ingress routes traffic for example.com. The stable-service handles most traffic. The canary-service is marked as a canary with 20% traffic using annotations. This means 20% of users go to the canary version for testing, while 80% use the stable version.

Commands

This command creates or updates the Ingress resource that controls traffic routing for A/B testing between stable and canary services.

Terminal

kubectl apply -f ingress-ab-testing.yaml

Expected Output

ingress.networking.k8s.io/example-ingress created

This command checks that the Ingress resource is created and shows its current status and rules.

Terminal

kubectl get ingress example-ingress

Expected Output

NAME              CLASS    HOSTS         ADDRESS        PORTS   AGE

example-ingress   <none>   example.com   192.168.99.100 80      10s

This command shows detailed information about the Ingress, including annotations and backend services used for routing.

Terminal

kubectl describe ingress example-ingress

Expected Output

Name:             example-ingress

Namespace:        default

Address:          192.168.99.100

Default backend:  <none>

Rules:

  Host        Path  Backends

  ----        ----  --------

  example.com

              /   stable-service:80 (10.0.0.1:80)

              /   canary-service:80 (10.0.0.2:80)

Annotations:

  nginx.ingress.kubernetes.io/canary: true

  nginx.ingress.kubernetes.io/canary-weight: 20

Events:          <none>

This command verifies that the canary version pods are running and ready to receive traffic.

Terminal

kubectl get pods -l app=canary-service

Expected Output

NAME                             READY   STATUS    RESTARTS   AGE

canary-service-5d8f7f7d7b-abc12 1/1     Running   0          5m

Key Concept

If you remember nothing else from this pattern, remember: Kubernetes Ingress can split traffic between stable and canary services using annotations to enable safe A/B testing.

Common Mistakes

Not adding the canary annotations to the Ingress resource

Without annotations, all traffic goes to the stable service, so no A/B testing happens.

Always add 'nginx.ingress.kubernetes.io/canary: "true"' and 'nginx.ingress.kubernetes.io/canary-weight' annotations to enable traffic splitting.

Using the same path for both stable and canary without specifying canary annotations

Ingress will route all traffic to the first matching backend, ignoring the canary service.

Use canary annotations and weights to tell Ingress how to split traffic between services.

Not verifying that canary pods are running before applying Ingress

Traffic sent to canary service will fail if pods are not ready, causing errors for users.

Check canary pods are running and ready with 'kubectl get pods -l app=canary-service' before routing traffic.

Summary

Create an Ingress resource with canary annotations to split traffic between stable and canary services.

Apply the Ingress and verify it is created and configured correctly.

Check that canary pods are running to safely receive test traffic.

Use annotations to control the percentage of traffic sent to the canary version.

Practice

1. What is the main purpose of using A/B testing with Kubernetes Ingress?

easy

A. To monitor CPU usage of pods

B. To increase the number of pods automatically

C. To split user traffic between different versions of an application

D. To backup Kubernetes cluster data

Solution

1. Step 1: Understand A/B testing concept in Kubernetes

   A/B testing with Ingress is used to route traffic between different app versions to test new features safely.

2. Step 2: Identify the purpose of Ingress in traffic management

   Ingress controls how external traffic reaches services, enabling traffic splitting for A/B testing.

3. Final Answer:

   To split user traffic between different versions of an application -> Option C

4. Quick Check:

   A/B testing = traffic split [OK]

Hint: A/B testing means splitting traffic between app versions [OK]

Common Mistakes:

• Confusing A/B testing with scaling pods
• Thinking Ingress is for monitoring only
• Assuming Ingress handles backups

2. Which annotation is commonly used in Kubernetes Ingress to split traffic by percentage for A/B testing?

easy

A. nginx.ingress.kubernetes.io/canary-weight

B. nginx.ingress.kubernetes.io/rewrite-target

C. nginx.ingress.kubernetes.io/ssl-redirect

D. nginx.ingress.kubernetes.io/proxy-body-size

Solution

1. Step 1: Identify annotations for traffic splitting

   The annotation 'nginx.ingress.kubernetes.io/canary-weight' is used to specify the percentage of traffic sent to the canary version.

2. Step 2: Differentiate from other annotations

   Other annotations like rewrite-target or ssl-redirect serve different purposes unrelated to traffic splitting.

3. Final Answer:

   nginx.ingress.kubernetes.io/canary-weight -> Option A

4. Quick Check:

   Canary weight = traffic percentage [OK]

Hint: Look for 'canary-weight' to split traffic by percent [OK]

Common Mistakes:

• Using rewrite-target for traffic splitting
• Confusing SSL redirect with traffic control
• Ignoring canary annotations

3. Given this Ingress snippet for A/B testing, what percentage of traffic goes to the canary service?

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: stable-service
            port:
              number: 80
      - path: /
        pathType: Prefix
        backend:
          service:
            name: canary-service
            port:
              number: 80

medium

A. 70%

B. 30%

C. 50%

D. 100%

Solution

1. Step 1: Read the canary-weight annotation

   The annotation 'nginx.ingress.kubernetes.io/canary-weight' is set to "30", meaning 30% of traffic goes to canary.

2. Step 2: Understand traffic split logic

   Traffic is split by percentage; 30% to canary-service, remaining 70% to stable-service.

3. Final Answer:

   30% -> Option B

4. Quick Check:

   Canary weight = 30% traffic [OK]

Hint: Check canary-weight value for traffic percent [OK]

Common Mistakes:

• Assuming canary gets 70% instead of 30%
• Confusing service names with traffic percentages
• Ignoring canary annotation

4. You configured this Ingress for A/B testing but all traffic goes to the stable service only. What is the likely error?

metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules:
  - host: test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: stable-service
            port:
              number: 80

medium

A. Missing canary backend path in spec

B. Incorrect canary-weight value format

C. Host name is invalid

D. Stable service port is wrong

Solution

1. Step 1: Check Ingress spec for canary backend

   The spec only has one backend for stable-service; no path defined for canary-service.

2. Step 2: Understand traffic routing requirements

   For canary traffic to work, a separate path with canary annotations and backend service must be defined.

3. Final Answer:

   Missing canary backend path in spec -> Option A

4. Quick Check:

   Canary needs separate backend path [OK]

Hint: Ensure canary backend path exists in Ingress spec [OK]

Common Mistakes:

• Assuming canary-weight alone routes traffic
• Ignoring missing canary backend path
• Blaming host or port without checking paths

5. You want to route 20% of users with header X-User-Type: beta to the canary service and the rest to stable. Which Ingress annotation setup achieves this?

hard

A. Use nginx.ingress.kubernetes.io/canary-by-cookie: "beta"

B. Use nginx.ingress.kubernetes.io/canary-weight: "20" only

C. Use nginx.ingress.kubernetes.io/canary: "false"

D. Use nginx.ingress.kubernetes.io/canary-by-header: "X-User-Type" and nginx.ingress.kubernetes.io/canary-by-header-value: "beta"

Solution

1. Step 1: Identify header-based routing annotations

   To route traffic based on header, use 'canary-by-header' and 'canary-by-header-value' annotations.

2. Step 2: Understand difference from weight-based routing

   Weight-based routing splits traffic by percentage regardless of headers; header-based routing targets specific users.

3. Final Answer:

   Use nginx.ingress.kubernetes.io/canary-by-header: "X-User-Type" and nginx.ingress.kubernetes.io/canary-by-header-value: "beta" -> Option D

4. Quick Check:

   Header-based routing uses canary-by-header annotations [OK]

Hint: Use canary-by-header and canary-by-header-value for header routing [OK]

Common Mistakes:

• Using only canary-weight for header routing
• Confusing cookie-based routing with header routing
• Setting canary to false disables routing