Monitoring NLP models - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to import the library used for monitoring model performance.

NLP

import [1]

Drag options to blanks, or click blank then click option'

Anumpy

Btensorflow

Cprometheus_client

Dmatplotlib

Attempts:

3 left

💡 Hint

Common Mistakes

Importing unrelated libraries like numpy or matplotlib for monitoring.

The prometheus_client library is commonly used to collect and expose metrics for monitoring NLP models.

2fill in blank

medium

Complete the code to define a metric that counts prediction requests.

NLP

prediction_counter = [1]('prediction_requests_total', 'Total number of prediction requests')

Drag options to blanks, or click blank then click option'

AGauge

BCounter

CHistogram

DSummary

Attempts:

3 left

💡 Hint

Common Mistakes

Using Gauge which can go up and down, not ideal for counting requests.

A Counter is used to count events like prediction requests in monitoring.

3fill in blank

hard

Fix the error in the code to start the Prometheus metrics server on port 8000.

NLP

from prometheus_client import start_http_server

start_http_server([1])

Drag options to blanks, or click blank then click option'

A'8000'

Bport=8000

Cport='8000'

D8000

Attempts:

3 left

💡 Hint

Common Mistakes

Passing port as a string or with keyword argument causes errors.

The start_http_server function expects an integer port number, so 8000 without quotes is correct.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that tracks average prediction latency per model.

NLP

avg_latency = {model: [1] for model, times in latency_data.items() if [2] > 0}

Drag options to blanks, or click blank then click option'

Asum(times) / len(times)

Blen(times)

Clen(times) > 0

Dsum(times)

Attempts:

3 left

💡 Hint

Common Mistakes

Using sum(times) without dividing by count.

Not checking if len(times) > 0.

The average latency is calculated by dividing the sum of times by the count of times. The condition ensures we only include models with recorded latencies.

5fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters models with accuracy above 0.8 and maps model names to their accuracies.

NLP

high_accuracy = {: {BLANK_2}} for [2], [1] in accuracy_data.items() if [2] > 0.8

Drag options to blanks, or click blank then click option'

Amodel

Baccuracy

C{

D}

Attempts:

3 left

💡 Hint

Common Mistakes

Not using curly braces for dictionary comprehension.

Mixing up key and value variable names.

The dictionary comprehension starts with {, uses model as key and accuracy as value, and filters accuracies above 0.8.