Why serving architecture affects latency and cost in MLOps - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to select the serving architecture type.

MLOps

serving_architecture = "[1]"

Drag options to blanks, or click blank then click option'

Astreaming

Bbatch

Coffline

Dmanual

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing 'batch' which is slower and higher latency.

The streaming architecture is commonly used for real-time serving, which affects latency and cost.

2fill in blank

medium

Complete the code to set the latency threshold for serving.

MLOps

latency_threshold_ms = [1]

Drag options to blanks, or click blank then click option'

A1000

B5000

C100

D10000

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing too high latency values that don't meet real-time needs.

100 milliseconds is a common low latency target for real-time serving.

3fill in blank

hard

Fix the error in the code to calculate cost per request.

MLOps

cost_per_request = total_cost / [1]

Drag options to blanks, or click blank then click option'

Atotal_users

Btotal_latency

Ctotal_time

Dtotal_requests

Attempts:

3 left

💡 Hint

Common Mistakes

Dividing by latency or time instead of request count.

Cost per request is calculated by dividing total cost by total number of requests served.

4fill in blank

hard

Fill both blanks to create a dictionary mapping architecture to latency.

MLOps

latency_map = {"batch": [1], "streaming": [2]

Drag options to blanks, or click blank then click option'

A5000

B100

C1000

D50

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping latency values between batch and streaming.

Batch serving typically has latency around 1000 ms, streaming around 100 ms.

5fill in blank

hard

Fill all three blanks to create a cost dictionary with architecture, latency, and cost per request.

MLOps

cost_info = {"architecture": "[1]", "latency_ms": [2], "cost_per_request_usd": [3]

Drag options to blanks, or click blank then click option'

Astreaming

B100

C0.0005

Dbatch

Attempts:

3 left

💡 Hint

Common Mistakes

Using batch architecture with high latency and cost.

Streaming architecture with 100 ms latency and cost per request of 0.0005 USD is typical for low-latency serving.