Canary releases for model updates in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Canary Release Purpose

Why is a canary release used when updating machine learning models in production?

A To deploy the new model only in a testing environment without any user traffic

B To immediately replace the old model with the new one for all users to save time

C To run both old and new models in parallel and average their predictions for all users

D To gradually expose the new model to a small portion of users to monitor performance before full deployment

💡 Hint

Think about minimizing risk when introducing changes.

A canary release gradually exposes the new model to a small user group. This helps detect issues early without affecting all users.

💻 Command Output

intermediate

Interpreting Traffic Routing Command Output

You run a command to route traffic to the new model version in a canary release. What is the output of this command?

MLOps

kubectl rollout status deployment/model-v2
kubectl patch service model-service -p '{"spec":{"selector":{"version":"v2"}}}'

A

deployment "model-v2" rolled out with warnings
service selector unchanged

B

error: deployment "model-v2" not found
service patch failed due to syntax error

C

deployment "model-v2" successfully rolled out
service patched to route traffic to version v2

D

deployment "model-v2" rolled out
service deleted accidentally

💡 Hint

Check for success messages and correct patch syntax.

The command confirms the new model deployment is ready and updates the service selector to route traffic to the new version.

🔀 Workflow

advanced

Correct Order of Canary Release Steps

Arrange the steps in the correct order for performing a canary release of a new ML model version.

A 3,1,2,4

B 1,2,3,4

C 1,3,2,4

D 2,1,3,4

💡 Hint

Think about deploying first, then routing traffic, then monitoring.

The canary release starts by deploying the new version, then routing a small traffic portion, monitoring results, and finally adjusting traffic or rolling back.

❓ Troubleshoot

advanced

Identifying Issue in Canary Release Traffic Routing

After deploying a new model version with a canary release, users report no change in predictions. What is the most likely cause?

A Traffic routing rules still direct 100% traffic to the old model version

B The new model version failed to build and was never deployed

C The monitoring system is not collecting data properly

D The new model version is slower but still serving predictions

💡 Hint

If users see no change, check where traffic is going.

If traffic routing is not updated, all user requests go to the old model, so predictions remain unchanged.

✅ Best Practice

expert

Best Practice for Canary Release Rollback

During a canary release, monitoring detects a significant drop in model accuracy. What is the best immediate action?

A Rollback the canary release by routing all traffic back to the stable model version

B Increase traffic to the new model to gather more data for analysis

C Ignore the drop and continue rollout to all users

D Deploy a third model version immediately without rollback

💡 Hint

When performance drops, safety first.

Rolling back quickly to the stable version prevents poor user experience and allows time to fix the new model.