Canary releases for model updates in MLOps - Commands & Configuration

Introduction

When you update a machine learning model, you want to make sure the new version works well before giving it to all users. Canary releases let you send the new model to a small group first, so you can check it carefully and avoid big problems.

When you want to test a new model version with a small group of users before full deployment

When you want to compare performance between the old and new models in real time

When you want to reduce risk by gradually rolling out model updates

When you want to monitor new model behavior and rollback quickly if issues appear

When you want to collect feedback or metrics on a new model version without affecting everyone

Commands

This command starts serving the current stable model on port 1234 so users can send requests to it.

Terminal

mlflow models serve -m runs:/1234567890abcdef/model -p 1234

Expected Output

2024/06/01 12:00:00 INFO mlflow.models.cli: Starting MLflow model server for model at runs:/1234567890abcdef/model on port 1234

2024/06/01 12:00:00 INFO mlflow.models.cli: Listening on http://0.0.0.0:1234

→

-m - Specifies the model path to serve

→

-p - Sets the port number for the server

This command starts serving the new model version on port 1235 for canary testing with a small user group.

Terminal

mlflow models serve -m runs:/abcdef1234567890/model -p 1235

Expected Output

2024/06/01 12:05:00 INFO mlflow.models.cli: Starting MLflow model server for model at runs:/abcdef1234567890/model on port 1235

2024/06/01 12:05:00 INFO mlflow.models.cli: Listening on http://0.0.0.0:1235

→

-m - Specifies the new model path to serve

→

-p - Sets a different port number for canary server

Send a prediction request to the stable model server to check it is working.

Terminal

curl -X POST -H 'Content-Type: application/json' -d '{"data": [[5.1, 3.5, 1.4, 0.2]]}' http://localhost:1234/invocations

Expected Output

{"predictions": [0]}

Send the same prediction request to the canary model server to compare results.

Terminal

curl -X POST -H 'Content-Type: application/json' -d '{"data": [[5.1, 3.5, 1.4, 0.2]]}' http://localhost:1235/invocations

Expected Output

{"predictions": [0]}

After successful canary testing, this command promotes the new model version to production for all users.

Terminal

mlflow models transition-stage --model-name my-model --version 2 --stage Production

Expected Output

Model version '2' of 'my-model' transitioned to stage 'Production'.

→

--model-name - Specifies the registered model name

→

--version - Specifies the model version to promote

→

--stage - Sets the new stage for the model version

Key Concept

If you remember nothing else from this pattern, remember: deploy the new model to a small group first, monitor it closely, then promote it to all users only if it works well.

Code Example

MLOps

import mlflow
import requests

# Define URLs for stable and canary model servers
stable_url = 'http://localhost:1234/invocations'
canary_url = 'http://localhost:1235/invocations'

# Sample input data
input_data = {"data": [[5.1, 3.5, 1.4, 0.2]]}

# Send request to stable model
response_stable = requests.post(stable_url, json=input_data)
print('Stable model prediction:', response_stable.json())

# Send request to canary model
response_canary = requests.post(canary_url, json=input_data)
print('Canary model prediction:', response_canary.json())

# Example of logging a metric during canary testing
mlflow.start_run(run_name='canary_test')
mlflow.log_metric('canary_accuracy', 0.95)
mlflow.end_run()
print('Metric logged for canary test')

Common Mistakes

Deploying the new model directly to all users without testing

This can cause widespread errors or bad predictions if the new model has issues.

Use canary releases to test the new model with a small group before full rollout.

Not monitoring the canary model's performance during testing

Without monitoring, you might miss problems and deploy a faulty model.

Collect metrics and logs during canary testing to catch issues early.

Using the same port or endpoint for both stable and canary models

This causes conflicts and makes it impossible to route traffic correctly.

Serve stable and canary models on different ports or endpoints.

Summary

Start serving the stable model on one port for all users.

Serve the new model version on a different port for a small group (canary).

Send test requests to both models to compare predictions.

Monitor canary model performance and log metrics.

Promote the new model to production only after successful testing.