Model validation gates in MLOps - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to define a validation gate that checks if model accuracy is above 0.8.

MLOps

validation_gate = ModelValidationGate(metric='accuracy', threshold=[1])

Drag options to blanks, or click blank then click option'

A0.5

B1.0

C0.8

D0.3

Attempts:

3 left

💡 Hint

Common Mistakes

Setting the threshold too low or too high, like 0.3 or 1.0.

The threshold for the validation gate should be set to 0.8 to ensure the model accuracy is above 80%.

2fill in blank

medium

Complete the code to add a validation gate that blocks deployment if the model's F1 score is below 0.75.

MLOps

validation_gate = ModelValidationGate(metric='f1_score', threshold=[1], comparison='greater')

Drag options to blanks, or click blank then click option'

A0.75

B0.9

C0.5

D0.85

Attempts:

3 left

💡 Hint

Common Mistakes

Using a threshold that is too high or too low for F1 score validation.

The threshold should be 0.75 to block deployment if the F1 score is less than 75%.

3fill in blank

hard

Fix the error in the validation gate code to correctly compare the model's loss metric.

MLOps

validation_gate = ModelValidationGate(metric='loss', threshold=[1], comparison='greater')

Drag options to blanks, or click blank then click option'

A0.05

B0.1

C0.01

D0.5

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'greater' comparison for loss metric instead of 'less'.

For loss, the comparison should be 'less' to block if loss is greater than threshold. Here, threshold 0.5 with 'greater' is incorrect, so the fix is to set threshold to 0.5 and change comparison to 'less' (not shown here). The blank is the threshold value 0.5.

4fill in blank

hard

Fill both blanks to create a validation gate that checks if precision is at least 0.85.

MLOps

validation_gate = ModelValidationGate(metric=[1], threshold=[2], comparison='greater')

Drag options to blanks, or click blank then click option'

A'precision'

B0.85

C0.75

D'recall'

Attempts:

3 left

💡 Hint

Common Mistakes

Confusing precision with recall or using wrong threshold values.

The metric should be 'precision' and the threshold 0.85 to ensure precision is at least 85%.

5fill in blank

hard

Fill all three blanks to define a validation gate that blocks deployment if recall is below 0.7 and uses 'less' comparison.

MLOps

validation_gate = ModelValidationGate(metric=[1], threshold=[2], comparison=[3])

Drag options to blanks, or click blank then click option'

A'recall'

B0.7

C'less'

D'greater'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'greater' comparison instead of 'less' for recall threshold.

The metric is 'recall', threshold 0.7, and comparison 'less' to block if recall is below 70%.