Streaming responses to users in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use streaming responses in AI chat applications?

Which of the following best explains why streaming responses are used in AI chat applications?

A To send parts of the response as they are generated, reducing wait time and improving user experience.

B To store the response on the server for later retrieval by the user.

C To send the entire response only after the AI finishes generating it, ensuring completeness.

D To compress the response data to save bandwidth during transmission.

💡 Hint

Think about how users feel when they wait for a long answer to appear all at once.

Streaming responses send parts of the answer as soon as they are ready, so users start seeing the reply immediately, making the interaction feel faster and smoother.

❓ Predict Output

intermediate

Output of streaming response simulation code

What will be printed by the following Python code simulating streaming AI responses?

Prompt Engineering / GenAI

import time
responses = ['Hello', ', ', 'how ', 'can ', 'I ', 'help ', 'you?']
for part in responses:
    print(part, end='', flush=True)
    time.sleep(0.1)
print('\nDone')

A Hello, how can I help you?\nDone

B Hello\n, \nhow \ncan \nI \nhelp \nyou?\nDone

C Hello, how can I help you? Done

D Hello, how can I help you?

💡 Hint

Look at how print uses end='' and flush=True.

The code prints each part without a newline, flushing output immediately, so the full sentence appears on one line, then 'Done' on the next line.

❓ Model Choice

advanced

Choosing a model architecture for streaming text generation

Which model architecture is best suited for generating streaming text responses token-by-token in real time?

A Autoencoder trained for data compression

B Convolutional Neural Network (CNN) trained for image classification

C Recurrent Neural Network (RNN) or Transformer decoder that generates tokens sequentially

D Feedforward Neural Network with fixed-size input and output

💡 Hint

Think about models that generate sequences one piece at a time.

RNNs and Transformer decoders generate text token-by-token, allowing streaming output as each token is produced.

❓ Metrics

advanced

Evaluating streaming response quality

Which metric is most appropriate to evaluate the quality of streaming text responses from an AI model?

A Silhouette score for clustering quality

B Confusion matrix of classification labels

C Mean Squared Error (MSE) between predicted and true token embeddings

D BLEU score comparing generated tokens to reference text

💡 Hint

Consider metrics that compare generated text to expected text.

BLEU score measures how closely generated text matches reference text, making it suitable for evaluating text generation quality.

🔧 Debug

expert

Identifying the cause of delayed streaming output

An AI chat app uses streaming to send tokens as they are generated. However, users report that the entire response appears only after a long delay. Which is the most likely cause?

A The model generates tokens in parallel and streams them immediately.

B The model generates tokens one by one but the server buffers output and sends it only after completion.

C The client displays tokens as soon as they arrive from the server.

D The network connection is very fast and stable.

💡 Hint

Think about where buffering might happen in the streaming pipeline.

If the server buffers all tokens before sending, streaming benefits are lost and users see the full response only after generation finishes.