Latency optimization in Prompt Engineering / GenAI - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to measure the latency of a function call using time module.

Prompt Engineering / GenAI

import time
start = time.[1]()
result = my_function()
end = time.time()
latency = end - start
print(f"Latency: {latency} seconds")

Drag options to blanks, or click blank then click option'

Asleep

Bperf_counter

Ctime

Dprocess_time

Attempts:

3 left

💡 Hint

Common Mistakes

Using time.sleep() instead of a timer function.

Using time.time() which is less precise.

time.perf_counter() gives the most precise timer for measuring short durations like latency.

2fill in blank

medium

Complete the code to batch process inputs to reduce latency in model inference.

Prompt Engineering / GenAI

batch_size = [1]
inputs = get_inputs()
batched_inputs = [inputs[i:i+batch_size] for i in range(0, len(inputs), batch_size)]

Drag options to blanks, or click blank then click option'

A1

B1000

C32

D0

Attempts:

3 left

💡 Hint

Common Mistakes

Using batch size 1 which does not improve latency.

Using batch size 0 which causes errors.

A batch size of 32 is a common choice to balance throughput and latency in inference.

3fill in blank

hard

Fix the error in the code to asynchronously run multiple inference calls to reduce latency.

Prompt Engineering / GenAI

import asyncio

async def infer_async(input):
    return model.predict(input)

async def main():
    tasks = [infer_async(i) for i in inputs]
    results = await asyncio.[1](tasks)
    print(results)

asyncio.run(main())

Drag options to blanks, or click blank then click option'

Agather

Bwait

Crun

Dsleep

Attempts:

3 left

💡 Hint

Common Mistakes

Using asyncio.wait which returns futures, not results directly.

Using asyncio.run inside async function causing errors.

asyncio.gather runs multiple async tasks concurrently and collects their results.

4fill in blank

hard

Fill both blanks to create a dictionary comprehension that filters features with latency less than threshold.

Prompt Engineering / GenAI

latency_dict = {feature: latency for feature, latency in features_latency.items() if latency [1] [2]

Drag options to blanks, or click blank then click option'

A<

B>

C0.05

D0.5

Attempts:

3 left

💡 Hint

Common Mistakes

Using '>' instead of '<' which filters wrong features.

Using too low or too high threshold values.

We want features with latency less than 0.5 seconds, so use '<' and 0.5.

5fill in blank

hard

Fill all three blanks to create a dictionary comprehension that maps model names to their average latency if latency is above threshold.

Prompt Engineering / GenAI

avg_latency = {model[1]: sum(times)/len(times) for model, times in latency_data.items() if sum(times)/len(times) [2] [3]

Drag options to blanks, or click blank then click option'

A.upper()

B>

C0.1

D.lower()

Attempts:

3 left

💡 Hint

Common Mistakes

Using .upper() instead of .lower() causing inconsistent keys.

Using '<' instead of '>' in the condition.

We convert model names to lowercase, and filter average latency greater than 0.1 seconds.