Data drift detection in MLOps - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary goal of data drift detection in MLOps?

Data drift detection is crucial in machine learning operations. What is its main purpose?

A To automate the deployment of new model versions

B To improve the training speed of machine learning models

C To monitor hardware resource usage during model training

D To identify changes in input data distribution that may affect model performance

💡 Hint

Think about why monitoring data quality over time matters for a model's accuracy.

Data drift detection helps spot when the data your model sees changes from what it was trained on, which can reduce accuracy.

💻 Command Output

intermediate

Output of a data drift detection command

Given the following command output from a data drift tool, what does it indicate?

{"feature": "age", "drift_detected": true, "p_value": 0.01}

A The feature 'age' has statistically significant drift with p-value 0.01

B The feature 'age' was removed from the dataset

C The model accuracy improved due to drift in 'age'

D No drift detected for feature 'age' because p-value is above 0.05

💡 Hint

Recall that a p-value below 0.05 usually means significant change.

A p-value of 0.01 means the change in 'age' distribution is statistically significant, so drift is detected.

❓ Configuration

advanced

Configuring a data drift detection threshold

Which configuration snippet correctly sets a data drift detection threshold to trigger alerts when p-value is below 0.05?

A

drift_detection:
  threshold: '0.05'
  alert: false

B

drift_detection:
  threshold: 0.05
  alert: true

C

drift_detection:
  threshold: 5
  alert: true

D

drift_detection:
  alert: true
  threshold: 0.5

💡 Hint

Threshold should be a decimal number representing p-value cutoff.

Setting threshold to 0.05 means alerts trigger when p-value is less than 0.05, indicating drift.

🔀 Workflow

advanced

Correct sequence for data drift detection workflow

What is the correct order of steps in a typical data drift detection workflow?

A 1,2,3,4

B 1,3,2,4

C 2,1,3,4

D 3,1,2,4

💡 Hint

Think about the natural flow from data collection to action.

First collect data, then compare distributions, alert if drift, then update model if needed.

❓ Troubleshoot

expert

Why does the data drift detection tool report no drift despite model accuracy dropping?

A deployed model's accuracy dropped significantly, but the data drift detection tool reports no drift. What is the most likely cause?

A The model was retrained recently, so drift is ignored

B The drift detection tool is malfunctioning and needs a restart

C The drift detection only monitors input features, but the issue is with label distribution shift

D The data drift detection threshold is set too low, causing false positives

💡 Hint

Consider what types of drift affect model accuracy but might not be detected by input data checks.

Data drift tools often monitor input features only; label distribution changes (label drift) can reduce accuracy without input drift.