GPU vs CPU inference tradeoffs in MLOps - Interactive Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to select the device for inference based on availability.

MLOps

device = 'cuda' if torch.cuda.is_available() else [1]

Drag options to blanks, or click blank then click option'

A'cpu'

B'tpu'

C'gpu'

D'fpga'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'gpu' as fallback device

Using 'tpu' or 'fpga' without hardware support

When GPU is not available, the CPU is used for inference.

2fill in blank

medium

Complete the code to set batch size for CPU inference to avoid overload.

MLOps

batch_size = [1] if device == 'cpu' else 64

Drag options to blanks, or click blank then click option'

A16

B512

C256

D128

Attempts:

3 left

💡 Hint

Common Mistakes

Using large batch sizes like 128 or 256 on CPU causing slowdowns

Smaller batch sizes like 16 help prevent CPU overload during inference.

3fill in blank

hard

Fix the error in the code that measures inference time on CPU.

MLOps

start = time.time()
output = model(input.to([1]))
end = time.time()

Drag options to blanks, or click blank then click option'

A'gpu'

B'cpu'

C'cuda'

D'tpu'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'cuda' or 'gpu' when running on CPU causing runtime errors

Input tensor must be moved to CPU device for CPU inference.

4fill in blank

hard

Fill both blanks to create a dictionary showing inference speed tradeoffs.

MLOps

inference_speed = {'CPU': [1], 'GPU': [2]  # in milliseconds

Drag options to blanks, or click blank then click option'

A100

B10

C50

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping CPU and GPU speeds

Using equal speeds for both devices

CPU inference is slower (100 ms) compared to GPU (5 ms) for the same model.

5fill in blank

hard

Fill all three blanks to filter models suitable for CPU inference with low memory.

MLOps

suitable_models = {m: mem for m, mem in models.items() if mem [1] 4 and 'light' [2] m and mem [3] 1}

Drag options to blanks, or click blank then click option'

A<=

Bin

C>=

D==

Attempts:

3 left

💡 Hint

Common Mistakes

Using wrong comparison operators

Using '==' instead of 'in' for substring check

We select models with memory less or equal 4GB, name containing 'light', and memory greater or equal 1GB.