Concept drift detection in MLOps - Mini Project: Build & Apply

Concept Drift Detection in Machine Learning

📖 Scenario: You are working as a machine learning engineer in a company that deploys models to predict customer behavior. Over time, the data your model sees can change, which may cause the model to perform worse. This change is called concept drift. Detecting concept drift early helps keep the model accurate and reliable.

🎯 Goal: Build a simple Python program that detects concept drift by comparing the distribution of new data with the original training data using a threshold.

📋 What You'll Learn

Create a dictionary called training_data_distribution with exact counts for categories

Create a variable called drift_threshold with the exact value 0.2

Write a function called detect_drift that takes two dictionaries and returns true if drift is detected

Print the result of calling detect_drift with training_data_distribution and new_data_distribution

💡 Why This Matters

🌍 Real World

Concept drift detection is crucial in real-world machine learning systems where data changes over time, such as fraud detection, recommendation systems, and customer behavior prediction.

💼 Career

Understanding and implementing concept drift detection helps machine learning engineers and MLOps professionals maintain model accuracy and reliability in production environments.

1

Create the training data distribution

Create a dictionary called training_data_distribution with these exact entries: 'A': 50, 'B': 30, 'C': 20.

MLOps

# Create the training data distribution dictionary
# Your code here

Need a hint?

Use curly braces {} to create a dictionary with keys 'A', 'B', and 'C' and their counts.

2

Set the drift detection threshold

Create a variable called drift_threshold and set it to the float value 0.2.

MLOps

training_data_distribution = {'A': 50, 'B': 30, 'C': 20}
# Set the drift detection threshold
# Your code here

Need a hint?

Assign the value 0.2 to the variable drift_threshold.

3

Write the concept drift detection function

Write a function called detect_drift that takes two dictionaries: original and new. It should calculate the total absolute difference in proportions for keys 'A', 'B', and 'C'. Return true if this difference is greater than or equal to drift_threshold, otherwise false. Use the formula: difference = sum of absolute differences of (new[key]/new_total) and (original[key]/original_total) for each key.

MLOps

training_data_distribution = {'A': 50, 'B': 30, 'C': 20}
drift_threshold = 0.2

# Write the detect_drift function below
# Your code here

Need a hint?

Calculate proportions by dividing counts by total counts. Sum absolute differences. Compare with drift_threshold.

4

Test and print the drift detection result

Create a dictionary called new_data_distribution with these exact entries: 'A': 40, 'B': 35, 'C': 25. Then print the result of calling detect_drift(training_data_distribution, new_data_distribution).

MLOps

training_data_distribution = {'A': 50, 'B': 30, 'C': 20}
drift_threshold = 0.2

def detect_drift(original, new):
    original_total = sum(original.values())
    new_total = sum(new.values())
    difference = 0
    for key in ['A', 'B', 'C']:
        original_prop = original[key] / original_total
        new_prop = new[key] / new_total
        difference += abs(new_prop - original_prop)
    return difference >= drift_threshold

# Create new_data_distribution and print the drift detection result
# Your code here

Need a hint?

Use the exact dictionary for new_data_distribution. Call detect_drift with the two dictionaries and print the result.