Prediction distribution monitoring in MLOps - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the monitoring library.

MLOps

import [1]

Drag options to blanks, or click blank then click option'

Aprometheus_client

Btensorflow

Cnumpy

Dmatplotlib

Attempts:

3 left

💡 Hint

Common Mistakes

Importing unrelated libraries like tensorflow or numpy for monitoring.

The prometheus_client library is commonly used for monitoring metrics in ML systems.

2fill in blank

medium

Complete the code to define a gauge metric for prediction distribution.

MLOps

prediction_gauge = Gauge('[1]', 'Distribution of model predictions')

Drag options to blanks, or click blank then click option'

Amodel_accuracy

Binput_data_count

Ctraining_loss

Dprediction_distribution

Attempts:

3 left

💡 Hint

Common Mistakes

Using unrelated metric names like accuracy or loss.

The gauge name should reflect the metric being tracked, here it is prediction_distribution.

3fill in blank

hard

Fix the error in updating the gauge with the latest prediction value.

MLOps

prediction_gauge.[1](latest_prediction)

Drag options to blanks, or click blank then click option'

Aset

Bset_value

Cupdate

Dincrement

Attempts:

3 left

💡 Hint

Common Mistakes

Using non-existent methods like set_value or update.

The correct method to update a Gauge metric value is set.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters predictions above a threshold.

MLOps

{pred: count for pred, count in prediction_counts.items() if pred [1] [2]

Drag options to blanks, or click blank then click option'

A>

B0.5

C<

D1.0

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong comparison operators or thresholds.

The comprehension filters predictions greater than 0.5.

5fill in blank

hard

Fill all three blanks to create a monitoring alert rule for prediction drift.

MLOps

alert_rule = {
  'alert': '[1]',
  'expr': 'abs(prediction_mean - baseline_mean) [2] [3]',
  'for': '5m'
}

Drag options to blanks, or click blank then click option'

APredictionDriftHigh

B>

C0.1

DPredictionAccuracyLow

Attempts:

3 left

💡 Hint

Common Mistakes

Using unrelated alert names or wrong operators.

The alert is named PredictionDriftHigh, triggers when the absolute difference is greater than 0.1.