Blue-green deployment for models in MLOps - Commands & Configuration

Introduction

Blue-green deployment helps you update machine learning models without stopping the service. It keeps two versions of the model running so users always get a working model while you test the new one.

When you want to update a model without causing downtime for users.

When you need to test a new model version with real traffic before fully switching.

When you want to quickly roll back to the old model if the new one has problems.

When you deploy models in production and want to avoid service interruptions.

When you want to compare performance between two model versions in real time.

Commands

Start serving the current stable model version (blue) on port 1234 so users can access it.

Terminal

mlflow models serve -m models:/my-model/blue -p 1234

Expected Output

2024/06/01 12:00:00 INFO mlflow.models.cli: Starting MLflow model server for model 'my-model' version 'blue' on port 1234

2024/06/01 12:00:00 INFO mlflow.models.cli: Listening on http://0.0.0.0:1234

→

-m - Specify the model URI to serve

→

-p - Set the port number for the server

Start serving the new model version (green) on a different port 1235 to test it without affecting users.

Terminal

mlflow models serve -m models:/my-model/green -p 1235

Expected Output

2024/06/01 12:01:00 INFO mlflow.models.cli: Starting MLflow model server for model 'my-model' version 'green' on port 1235

2024/06/01 12:01:00 INFO mlflow.models.cli: Listening on http://0.0.0.0:1235

→

-m - Specify the model URI to serve

→

-p - Set the port number for the server

Send a test prediction request to the green model to verify it works correctly before switching traffic.

Terminal

curl -X POST http://localhost:1235/invocations -H 'Content-Type: application/json' -d '{"data": [[5.1, 3.5, 1.4, 0.2]]}'

Expected Output

{"predictions": [0]}

Switch the production stage to the green model version so all users start using the new model.

Terminal

mlflow models transition --model-name my-model --stage Production --version green

Expected Output

Model 'my-model' version 'green' is now in stage 'Production'.

Stop serving the old blue model version after confirming the green version works well.

Terminal

mlflow models stop -m models:/my-model/blue

Expected Output

Stopped serving model 'my-model' version 'blue'.

→

-m - Specify the model URI to stop serving

Key Concept

If you remember nothing else from this pattern, remember: run two model versions side-by-side, test the new one, then switch traffic smoothly without downtime.

Code Example

MLOps

import mlflow
import requests
import json

# Serve blue model (assumed running externally)

# Serve green model (assumed running externally)

# Test green model prediction
input_data = {"data": [[5.1, 3.5, 1.4, 0.2]]}
response = requests.post("http://localhost:1235/invocations", headers={"Content-Type": "application/json"}, data=json.dumps(input_data))
print("Green model prediction response:", response.text)

# Transition production to green model
client = mlflow.tracking.MlflowClient()
client.transition_model_version_stage(name="my-model", version=1, stage="Production")
print("Switched production to green model version.")

Common Mistakes

Stopping the old model before verifying the new model works.

This causes downtime if the new model has errors or is not ready.

Always run both models simultaneously and test the new one before switching.

Serving both models on the same port.

Only one service can listen on a port, so the second server will fail to start.

Use different ports for blue and green model servers during testing.

Not switching the production stage in the model registry.

Traffic will continue going to the old model, so the new model won't be used.

Use the model registry to update the production stage to the new model version.

Summary

Start serving the current stable model (blue) on one port.

Serve the new model (green) on a different port to test it safely.

Send test requests to the green model to verify correctness.

Switch production traffic to the green model using the model registry.

Stop the old blue model after confirming the new model works well.

Practice

1. What is the main purpose of blue-green deployment in model updates?

easy

A. To run two models at the same time and combine their outputs

B. To switch traffic to a new model only after it is fully tested and ready

C. To update the model directly in the production environment without backup

D. To deploy models only during off-peak hours

Solution

1. Step 1: Understand blue-green deployment concept

   Blue-green deployment uses two separate environments to avoid downtime and risk during updates.

2. Step 2: Identify the key purpose

   The main goal is to switch traffic to the new model only after it is fully tested and ready, ensuring safety.

3. Final Answer:

   To switch traffic to a new model only after it is fully tested and ready -> Option B

4. Quick Check:

   Safe model update = A [OK]

Hint: Blue-green means switch only after testing new model [OK]

Common Mistakes:

• Thinking both models run and combine outputs
• Updating production without backup
• Deploying only during off-peak hours

2. Which command correctly switches traffic from the blue environment to the green environment in a Kubernetes service?

easy

A. kubectl set image deployment/model-deploy model=green-model:latest

B. kubectl delete deployment model-deploy-blue

C. kubectl rollout restart deployment/model-deploy-green

D. kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}'

Solution

1. Step 1: Understand traffic switching in Kubernetes

   Traffic is routed by the service selector labels pointing to the correct deployment environment.

2. Step 2: Identify the command that changes service selector to green

   The patch command updates the service selector to point to pods labeled with "env=green".

3. Final Answer:

   kubectl patch service model-service -p '{"spec":{"selector":{"env":"green"}}}' -> Option D

4. Quick Check:

   Change service selector = B [OK]

Hint: Patch service selector to green environment label [OK]

Common Mistakes:

• Restarting deployment does not switch traffic
• Setting image changes deployment but not traffic
• Deleting blue deployment before switch causes downtime

3. Given this simplified deployment script snippet, what will be the output after running it?

current_env = "blue"
new_env = "green"
if current_env == "blue":
    print(f"Switching traffic to {new_env} environment")
else:
    print(f"Switching traffic to {current_env} environment")

medium

A. Switching traffic to green environment

B. Switching traffic to undefined environment

C. Switching traffic to blue environment

D. No output

Solution

1. Step 1: Analyze the condition in the script

   The variable current_env is "blue", so the if condition is true.

2. Step 2: Determine the printed output

   Since current_env is "blue", it prints "Switching traffic to green environment" using new_env.

3. Final Answer:

   Switching traffic to green environment -> Option A

4. Quick Check:

   current_env == "blue" triggers green switch = C [OK]

Hint: Check if current_env is blue, then print green [OK]

Common Mistakes:

• Confusing current_env and new_env variables
• Assuming else branch runs when condition is true
• Ignoring f-string formatting

4. You tried to switch traffic to the green model environment but users still hit the blue model. What is the most likely error?

medium

A. The service selector was not updated to point to the green environment

B. The green model deployment was deleted accidentally

C. The blue environment pods crashed and restarted

D. The model version in green environment is outdated

Solution

1. Step 1: Understand traffic routing in blue-green deployment

   Traffic is controlled by the service selector labels pointing to the active environment.

2. Step 2: Identify why traffic still hits blue

   If users still hit blue, the service selector likely was not updated to green, so traffic stays on blue.

3. Final Answer:

   The service selector was not updated to point to the green environment -> Option A

4. Quick Check:

   Traffic routing depends on service selector = D [OK]

Hint: Check if service selector changed to green environment [OK]

Common Mistakes:

• Assuming green deployment deletion causes traffic to blue
• Confusing pod crashes with traffic routing
• Thinking model version affects routing directly

5. You want to implement blue-green deployment for a machine learning model with minimal downtime. Which sequence of steps is correct?

hard

A. Deploy new model to green, switch traffic immediately, then test and monitor

B. Delete blue environment, deploy new model to green, switch traffic, monitor performance

C. Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue

D. Deploy new model to blue environment, test it, switch service selector to blue, monitor, then delete green

Solution

1. Step 1: Deploy and test new model in green environment

   Deploying and testing in green ensures the new model works without affecting users.

2. Step 2: Switch traffic to green, monitor, then clean up blue

   Switching traffic only after testing reduces risk. Monitoring ensures stability before deleting blue.

3. Final Answer:

   Deploy new model to green environment, test it, switch service selector to green, monitor, then delete blue -> Option C

4. Quick Check:

   Test before switch, monitor after = A [OK]

Hint: Test green first, then switch traffic, then delete blue [OK]

Common Mistakes:

• Deleting blue before green is ready
• Switching traffic before testing
• Deploying new model to blue environment instead of green