Why Blue-green deployment for models in MLOps? - Purpose & Use Cases

The Big Idea

What if you could update your AI model without users ever noticing a thing?

The Scenario

Imagine you have a machine learning model running in production that users rely on every day. Now, you want to update it with a better version. Doing this manually means stopping the current model, swapping in the new one, and hoping everything works perfectly without breaking the service.

The Problem

This manual swap is risky and slow. If the new model has bugs or performs worse, users face errors or delays. Rolling back is complicated and can cause downtime. It's like changing a car's engine while driving--very stressful and error-prone.

The Solution

Blue-green deployment creates two identical environments: one running the current model (blue) and one ready with the new model (green). You switch traffic smoothly from blue to green without downtime. If something goes wrong, you can quickly switch back, keeping users happy and services stable.

Before vs After

✗ Before

stop current_model
start new_model
if error:
  rollback manually

✓ After

deploy new_model to green
switch traffic from blue to green
if issue:
  switch traffic back to blue

What It Enables

It enables seamless, safe model updates with zero downtime and instant rollback.

Real Life Example

A streaming service updates its recommendation model daily. Using blue-green deployment, users never see interruptions or bad recommendations during updates.

Key Takeaways

Manual model updates risk downtime and errors.

Blue-green deployment runs two environments for smooth switching.

It ensures safe updates and quick rollback without user impact.

Practice

1. What is the main purpose of blue-green deployment in model updates?

easy

A. To run two models at the same time and combine their outputs

B. To switch traffic to a new model only after it is fully tested and ready

C. To update the model directly in the production environment without backup

D. To deploy models only during off-peak hours

Solution

1. Step 1: Understand blue-green deployment concept

   Blue-green deployment uses two separate environments to avoid downtime and risk during updates.

2. Step 2: Identify the key purpose

   The main goal is to switch traffic to the new model only after it is fully tested and ready, ensuring safety.

3. Final Answer:

   To switch traffic to a new model only after it is fully tested and ready -> Option B

4. Quick Check:

   Safe model update = A [OK]

Hint: Blue-green means switch only after testing new model [OK]

Common Mistakes:

• Thinking both models run and combine outputs
• Updating production without backup
• Deploying only during off-peak hours

2. Which command correctly switches traffic from the blue environment to the green environment in a Kubernetes service?

easy

A. kubectl set image deployment/model-deploy model=green-model:latest

B. kubectl delete deployment model-deploy-blue

C. kubectl rollout restart deployment/model-deploy-green

D. kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}'

Solution

1. Step 1: Understand traffic switching in Kubernetes

   Traffic is routed by the service selector labels pointing to the correct deployment environment.

2. Step 2: Identify the command that changes service selector to green

   The patch command updates the service selector to point to pods labeled with "env=green".

3. Final Answer:

   kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}' -> Option D

4. Quick Check:

   Change service selector = B [OK]

Hint: Patch service selector to green environment label [OK]

Common Mistakes:

• Restarting deployment does not switch traffic
• Setting image changes deployment but not traffic
• Deleting blue deployment before switch causes downtime

3. Given this simplified deployment script snippet, what will be the output after running it?

current_env = "blue"
new_env = "green"
if current_env == "blue":
    print(f"Switching traffic to {new_env} environment")
else:
    print(f"Switching traffic to {current_env} environment")

medium

A. Switching traffic to green environment

B. Switching traffic to undefined environment

C. Switching traffic to blue environment

D. No output

Solution

1. Step 1: Analyze the condition in the script

   The variable current_env is "blue", so the if condition is true.

2. Step 2: Determine the printed output

   Since current_env is "blue", it prints "Switching traffic to green environment" using new_env.

3. Final Answer:

   Switching traffic to green environment -> Option A

4. Quick Check:

   current_env == "blue" triggers green switch = C [OK]

Hint: Check if current_env is blue, then print green [OK]

Common Mistakes:

• Confusing current_env and new_env variables
• Assuming else branch runs when condition is true
• Ignoring f-string formatting

4. You tried to switch traffic to the green model environment but users still hit the blue model. What is the most likely error?

medium

A. The service selector was not updated to point to the green environment

B. The green model deployment was deleted accidentally

C. The blue environment pods crashed and restarted

D. The model version in green environment is outdated

Solution

1. Step 1: Understand traffic routing in blue-green deployment

   Traffic is controlled by the service selector labels pointing to the active environment.

2. Step 2: Identify why traffic still hits blue

   If users still hit blue, the service selector likely was not updated to green, so traffic stays on blue.

3. Final Answer:

   The service selector was not updated to point to the green environment -> Option A

4. Quick Check:

   Traffic routing depends on service selector = D [OK]

Hint: Check if service selector changed to green environment [OK]

Common Mistakes:

• Assuming green deployment deletion causes traffic to blue
• Confusing pod crashes with traffic routing
• Thinking model version affects routing directly

5. You want to implement blue-green deployment for a machine learning model with minimal downtime. Which sequence of steps is correct?

hard

A. Deploy new model to green, switch traffic immediately, then test and monitor

B. Delete blue environment, deploy new model to green, switch traffic, monitor performance

C. Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue

D. Deploy new model to blue environment, test it, switch service selector to blue, monitor, then delete green

Solution

1. Step 1: Deploy and test new model in green environment

   Deploying and testing in green ensures the new model works without affecting users.

2. Step 2: Switch traffic to green, monitor, then clean up blue

   Switching traffic only after testing reduces risk. Monitoring ensures stability before deleting blue.

3. Final Answer:

   Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue -> Option C

4. Quick Check:

   Test before switch, monitor after = A [OK]

Hint: Test green first, then switch traffic, then delete blue [OK]

Common Mistakes:

• Deleting blue before green is ready
• Switching traffic before testing
• Deploying new model to blue environment instead of green