Data drift detection basics in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Data Drift

What does data drift mean in the context of machine learning models?

A A change in the input data distribution over time that can affect model performance

B A sudden failure of the model due to hardware issues

C An increase in the size of the training dataset

D A change in the model's architecture during training

💡 Hint

Think about how the data the model sees might change after deployment.

Data drift refers to changes in the input data's statistical properties over time. This can cause the model to perform worse because it was trained on different data.

💻 Command Output

intermediate

Detecting Data Drift with a Python Library

Given the following Python code snippet using the alibi-detect library to detect data drift, what will be the output if the new data distribution is significantly different from the reference data?

MLOps

from alibi_detect.cd import KSDrift
import numpy as np

# Reference data: normal distribution
ref_data = np.random.normal(0, 1, 1000)

# New data: shifted mean
new_data = np.random.normal(2, 1, 1000)

cd = KSDrift(ref_data)
drift_result = cd.predict(new_data)
print(drift_result['data']['is_drift'])

A None

B False

C Raises TypeError

D True

💡 Hint

KSDrift uses the Kolmogorov-Smirnov test to detect if distributions differ.

The new data has a different mean, so the KS test will detect drift and return True.

❓ Configuration

advanced

Configuring a Data Drift Monitoring Pipeline

Which of the following YAML configurations correctly sets up a data drift monitoring job that runs daily and alerts if drift is detected?

A

schedule: hourly
monitor:
  type: data_drift
  alert: false
  threshold: 0.5

B

schedule: daily
monitor:
  type: data_drift
  alert: true
  threshold: 0.05

C

schedule: daily
monitor:
  type: data_drift
  alert: yes
  threshold: 0.01

D

schedule: daily
monitor:
  type: drift_detection
  alert: true
  threshold: 0.05

💡 Hint

Check for correct keys and valid values for alert and threshold.

Option B uses correct keys and values: 'daily' schedule, 'data_drift' type, alert enabled as boolean, and a reasonable threshold.

❓ Troubleshoot

advanced

Troubleshooting False Negatives in Data Drift Detection

A data drift detection system is not alerting even though the input data has clearly changed. Which of the following is the most likely cause?

A The monitoring job is running too frequently

B The model was retrained recently

C The drift detection threshold is set too high, making it insensitive to changes

D The input data format changed but the detection system ignores format

💡 Hint

Consider how sensitivity settings affect detection.

If the threshold is too high, small or moderate changes won't trigger alerts, causing false negatives.

🔀 Workflow

expert

Order of Steps in Data Drift Detection Workflow

Arrange the following steps in the correct order for a typical data drift detection workflow.

A 1,2,3,4

B 1,3,2,4

C 2,1,3,4

D 3,1,2,4

💡 Hint

Think about the logical flow from data collection to alerting.

The workflow starts with collecting reference data, then new data, followed by testing, and finally alerting if drift is found.