Evidently AI for monitoring in MLOps - Mini Project: Build & Apply

Evidently AI for Monitoring

📖 Scenario: You work as a data engineer in a company that runs machine learning models in production. Your team wants to monitor the quality of predictions over time to catch any problems early. You will use Evidently AI, a tool that helps track data and model quality metrics easily.

🎯 Goal: Build a simple Python script that loads sample prediction data, configures Evidently AI to monitor data drift, runs the monitoring analysis, and prints the report summary.

📋 What You'll Learn

Create a pandas DataFrame called reference_data with sample features and predictions.

Create a configuration variable called data_drift_report using Evidently AI's Report class with DataDriftProfile.

Run the run method on data_drift_report with reference_data as both reference and current data.

Print the JSON summary of the report using data_drift_report.json().

💡 Why This Matters

🌍 Real World

Monitoring machine learning models in production helps catch data or model quality issues early, preventing bad decisions or degraded user experience.

💼 Career

Data engineers and MLOps engineers use tools like Evidently AI to automate monitoring and ensure ML models stay reliable over time.

1

Create sample prediction data

Create a pandas DataFrame called reference_data with these exact columns and values: 'feature1' with values [10, 20, 30, 40, 50], 'feature2' with values [1, 2, 3, 4, 5], and 'prediction' with values [0, 1, 0, 1, 0].

MLOps

import pandas as pd
# Create the DataFrame reference_data with specified columns and values
# Your code here

Need a hint?

Use pd.DataFrame with a dictionary of lists for columns.

2

Configure Evidently AI data drift report

Import Report and DataDriftProfile from evidently. Then create a variable called data_drift_report and assign it a Report instance with the profile set to [DataDriftProfile()].

MLOps

import pandas as pd
reference_data = pd.DataFrame({
    'feature1': [10, 20, 30, 40, 50],
    'feature2': [1, 2, 3, 4, 5],
    'prediction': [0, 1, 0, 1, 0]
})

# Import Report and DataDriftProfile from evidently
# Create data_drift_report with Report(profile=[DataDriftProfile()])
# Your code here

Need a hint?

Use Report with profile=[DataDriftProfile()] to monitor data drift.

3

Run the data drift analysis

Use the run method on data_drift_report with reference_data as both the reference_data and current_data arguments.

MLOps

import pandas as pd
from evidently import Report
from evidently.profile_sections import DataDriftProfile

reference_data = pd.DataFrame({
    'feature1': [10, 20, 30, 40, 50],
    'feature2': [1, 2, 3, 4, 5],
    'prediction': [0, 1, 0, 1, 0]
})

data_drift_report = Report(profile=[DataDriftProfile()])

# Run data_drift_report.run with reference_data as both reference_data and current_data
# Your code here

Need a hint?

Call run on data_drift_report with the same DataFrame for both arguments.

4

Print the data drift report summary

Print the JSON summary of the data drift report by calling print(data_drift_report.json()).

MLOps

import pandas as pd
from evidently import Report
from evidently.profile_sections import DataDriftProfile

reference_data = pd.DataFrame({
    'feature1': [10, 20, 30, 40, 50],
    'feature2': [1, 2, 3, 4, 5],
    'prediction': [0, 1, 0, 1, 0]
})

data_drift_report = Report(profile=[DataDriftProfile()])

data_drift_report.run(reference_data=reference_data, current_data=reference_data)

# Print the JSON summary of the report
# Your code here

Need a hint?

Use print(data_drift_report.json()) to show the report summary in JSON format.