Blue-green deployment for models in MLOps - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is blue-green deployment in the context of machine learning models?

Blue-green deployment is a technique where two identical environments (blue and green) are maintained. One environment runs the current model (blue), while the other (green) hosts the new model version. Traffic is switched to green after testing, minimizing downtime and risk.

beginner

Why is blue-green deployment useful for machine learning models?

It allows safe updates by running the new model alongside the old one. If issues arise, traffic can quickly switch back to the stable model, reducing risk and downtime.

beginner

What happens during the 'switch' phase in blue-green deployment?

The system redirects user requests from the old model environment (blue) to the new model environment (green), making the new model live without downtime.

intermediate

How can you test the new model in blue-green deployment before switching traffic?

You can run automated tests, monitor performance metrics, or route a small percentage of real traffic to the green environment to validate the new model's behavior.

intermediate

What is a key difference between blue-green deployment and canary deployment for models?

Blue-green switches all traffic at once between two environments, while canary deployment gradually shifts traffic to the new model to monitor impact over time.

In blue-green deployment, what does the 'green' environment represent?

A The current live model serving all traffic

B The new model version ready for testing and deployment

C A backup environment with no models

D An environment for data preprocessing only

The green environment hosts the new model version before it becomes live.

What is the main benefit of blue-green deployment for models?

A Allows zero downtime and quick rollback

B Reduces model training time

C Eliminates the need for testing

D Automatically improves model accuracy

Blue-green deployment minimizes downtime and enables quick rollback if the new model has issues.

During blue-green deployment, how do you verify the new model before switching all traffic?

A Ignore testing and switch immediately

B Train the model again

C Delete the blue environment

D Run tests and monitor metrics on the green environment

Testing and monitoring on the green environment ensures the new model works well before full deployment.

If the new model in green environment fails after switching, what is the best action?

A Switch traffic back to blue environment

B Train a new model immediately

C Ignore errors and continue

D Delete both environments

Switching back to the stable blue environment quickly restores service.

Which deployment method gradually shifts traffic to a new model instead of switching all at once?

A Blue-green deployment

B Shadow deployment

C Canary deployment

D Rolling deployment

Canary deployment gradually shifts traffic to monitor new model impact.

Explain the steps involved in blue-green deployment for machine learning models.

Think about how to update a model without stopping service.

Describe the advantages and potential challenges of using blue-green deployment for models.

Consider both benefits and what might make it harder to use.

Practice

1. What is the main purpose of blue-green deployment in model updates?

easy

A. To run two models at the same time and combine their outputs

B. To switch traffic to a new model only after it is fully tested and ready

C. To update the model directly in the production environment without backup

D. To deploy models only during off-peak hours

Solution

1. Step 1: Understand blue-green deployment concept

   Blue-green deployment uses two separate environments to avoid downtime and risk during updates.

2. Step 2: Identify the key purpose

   The main goal is to switch traffic to the new model only after it is fully tested and ready, ensuring safety.

3. Final Answer:

   To switch traffic to a new model only after it is fully tested and ready -> Option B

4. Quick Check:

   Safe model update = A [OK]

Hint: Blue-green means switch only after testing new model [OK]

Common Mistakes:

• Thinking both models run and combine outputs
• Updating production without backup
• Deploying only during off-peak hours

2. Which command correctly switches traffic from the blue environment to the green environment in a Kubernetes service?

easy

A. kubectl set image deployment/model-deploy model=green-model:latest

B. kubectl delete deployment model-deploy-blue

C. kubectl rollout restart deployment/model-deploy-green

D. kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}'

Solution

1. Step 1: Understand traffic switching in Kubernetes

   Traffic is routed by the service selector labels pointing to the correct deployment environment.

2. Step 2: Identify the command that changes service selector to green

   The patch command updates the service selector to point to pods labeled with "env=green".

3. Final Answer:

   kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}' -> Option D

4. Quick Check:

   Change service selector = B [OK]

Hint: Patch service selector to green environment label [OK]

Common Mistakes:

• Restarting deployment does not switch traffic
• Setting image changes deployment but not traffic
• Deleting blue deployment before switch causes downtime

3. Given this simplified deployment script snippet, what will be the output after running it?

current_env = "blue"
new_env = "green"
if current_env == "blue":
    print(f"Switching traffic to {new_env} environment")
else:
    print(f"Switching traffic to {current_env} environment")

medium

A. Switching traffic to green environment

B. Switching traffic to undefined environment

C. Switching traffic to blue environment

D. No output

Solution

1. Step 1: Analyze the condition in the script

   The variable current_env is "blue", so the if condition is true.

2. Step 2: Determine the printed output

   Since current_env is "blue", it prints "Switching traffic to green environment" using new_env.

3. Final Answer:

   Switching traffic to green environment -> Option A

4. Quick Check:

   current_env == "blue" triggers green switch = C [OK]

Hint: Check if current_env is blue, then print green [OK]

Common Mistakes:

• Confusing current_env and new_env variables
• Assuming else branch runs when condition is true
• Ignoring f-string formatting

4. You tried to switch traffic to the green model environment but users still hit the blue model. What is the most likely error?

medium

A. The service selector was not updated to point to the green environment

B. The green model deployment was deleted accidentally

C. The blue environment pods crashed and restarted

D. The model version in green environment is outdated

Solution

1. Step 1: Understand traffic routing in blue-green deployment

   Traffic is controlled by the service selector labels pointing to the active environment.

2. Step 2: Identify why traffic still hits blue

   If users still hit blue, the service selector likely was not updated to green, so traffic stays on blue.

3. Final Answer:

   The service selector was not updated to point to the green environment -> Option A

4. Quick Check:

   Traffic routing depends on service selector = D [OK]

Hint: Check if service selector changed to green environment [OK]

Common Mistakes:

• Assuming green deployment deletion causes traffic to blue
• Confusing pod crashes with traffic routing
• Thinking model version affects routing directly

5. You want to implement blue-green deployment for a machine learning model with minimal downtime. Which sequence of steps is correct?

hard

A. Deploy new model to green, switch traffic immediately, then test and monitor

B. Delete blue environment, deploy new model to green, switch traffic, monitor performance

C. Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue

D. Deploy new model to blue environment, test it, switch service selector to blue, monitor, then delete green

Solution

1. Step 1: Deploy and test new model in green environment

   Deploying and testing in green ensures the new model works without affecting users.

2. Step 2: Switch traffic to green, monitor, then clean up blue

   Switching traffic only after testing reduces risk. Monitoring ensures stability before deleting blue.

3. Final Answer:

   Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue -> Option C

4. Quick Check:

   Test before switch, monitor after = A [OK]

Hint: Test green first, then switch traffic, then delete blue [OK]

Common Mistakes:

• Deleting blue before green is ready
• Switching traffic before testing
• Deploying new model to blue environment instead of green