Prediction distribution monitoring in MLOps - Mini Project: Build & Apply

Prediction Distribution Monitoring

📖 Scenario: You work as a machine learning engineer. Your model makes predictions as probabilities for a binary classification task. To keep the model reliable, you want to monitor how the prediction probabilities change over time. This helps detect if the model starts behaving differently than before.

🎯 Goal: Build a simple Python script that stores a set of prediction probabilities, sets a threshold for alerting, calculates how many predictions fall outside the normal range, and prints the alert count.

📋 What You'll Learn

Create a list of prediction probabilities with exact values

Add a threshold variable for the minimum acceptable probability

Use a loop to count how many predictions are below the threshold

Print the count of predictions below the threshold

💡 Why This Matters

🌍 Real World

Monitoring prediction distributions helps detect when a machine learning model's behavior changes, which can indicate data drift or model degradation.

💼 Career

This skill is important for ML engineers and DevOps professionals who maintain reliable AI systems in production.

1

Create initial prediction probabilities list

Create a list called predictions with these exact float values: 0.95, 0.85, 0.60, 0.40, 0.30, 0.80, 0.20, 0.10

MLOps

# Create the list of prediction probabilities
# Your code here

Need a hint?

Use square brackets to create a list and separate values with commas.

2

Set the alert threshold

Add a variable called threshold and set it to the float value 0.5

MLOps

predictions = [0.95, 0.85, 0.60, 0.40, 0.30, 0.80, 0.20, 0.10]
# Set the threshold value
# Your code here

Need a hint?

Use a simple assignment statement to create the threshold variable.

3

Count predictions below threshold

Create a variable called alert_count and set it to 0. Then use a for loop with variable prob to iterate over predictions. Inside the loop, increase alert_count by 1 if prob is less than threshold

MLOps

predictions = [0.95, 0.85, 0.60, 0.40, 0.30, 0.80, 0.20, 0.10]
threshold = 0.5
# Count how many predictions are below the threshold
# Your code here

Need a hint?

Use a for loop and an if statement to check each probability.

4

Print the alert count

Write a print statement to display the value of alert_count

MLOps

predictions = [0.95, 0.85, 0.60, 0.40, 0.30, 0.80, 0.20, 0.10]
threshold = 0.5
alert_count = 0
for prob in predictions:
    if prob < threshold:
        alert_count += 1
# Print the alert count
# Your code here

Need a hint?

Use print(alert_count) to show the number of alerts.