Streaming in production in LangChain - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to enable streaming output in a LangChain LLM call.

LangChain

from langchain.llms import OpenAI
llm = OpenAI(streaming=[1])
response = llm("Hello, how are you?")

Drag options to blanks, or click blank then click option'

ANone

BFalse

C"yes"

DTrue

Attempts:

3 left

💡 Hint

Common Mistakes

Using a string instead of a boolean for streaming parameter.

Leaving streaming as False or None, which disables streaming.

Setting streaming=True enables the LLM to stream tokens as they are generated.

2fill in blank

medium

Complete the code to define a callback handler for streaming tokens in LangChain.

LangChain

from langchain.callbacks.base import BaseCallbackHandler

class StreamHandler(BaseCallbackHandler):
    def on_llm_new_token(self, token: str, **kwargs):
        print([1])

Drag options to blanks, or click blank then click option'

Akwargs

Bself

Ctoken

DNone

Attempts:

3 left

💡 Hint

Common Mistakes

Printing self or kwargs instead of the token string.

Forgetting to print anything inside the method.

The on_llm_new_token method receives each token as a string, which we print.

3fill in blank

hard

Fix the error in attaching the streaming callback handler to the OpenAI LLM.

LangChain

llm = OpenAI(streaming=True, callbacks=[[1]])

Drag options to blanks, or click blank then click option'

AStreamHandler()

BStreamHandler

CStreamHandler.on_llm_new_token

DStreamHandler.callback

Attempts:

3 left

💡 Hint

Common Mistakes

Passing the class name without parentheses.

Passing a method instead of an instance.

You must pass an instance of the handler class, not the class itself.

4fill in blank

hard

Fill both blanks to create a streaming LLM with a callback handler and run a prompt.

LangChain

llm = OpenAI(streaming=[1], callbacks=[[2]])
result = llm("Tell me a joke.")

Drag options to blanks, or click blank then click option'

ATrue

BFalse

CStreamHandler()

DStreamHandler

Attempts:

3 left

💡 Hint

Common Mistakes

Setting streaming to False disables streaming.

Passing the class name instead of an instance for callbacks.

Streaming must be True and the callback must be an instance of StreamHandler.

5fill in blank

hard

Fill all three blanks to define a streaming callback handler that collects tokens, then print the full response.

LangChain

class CollectHandler(BaseCallbackHandler):
    def __init__(self):
        self.text = ""
    def on_llm_new_token(self, token: str, **kwargs):
        self.text += [1]

handler = CollectHandler()
llm = OpenAI(streaming=[2], callbacks=[[3]])
response = llm("Say something nice.")
print(handler.text)

Drag options to blanks, or click blank then click option'

Atoken

B" "

Chandler

DTrue

Attempts:

3 left

💡 Hint

Common Mistakes

Adding a space string instead of the token.

Passing the class name instead of the instance as callback.

Setting streaming to False.

Add each token to self.text, enable streaming=True, and pass the handler instance.

Practice

1. What does enabling streaming=True in LangChain do?

easy

A. It sends tokens immediately as they are generated.

B. It delays token sending until the entire response is ready.

C. It disables callbacks for token processing.

D. It caches all tokens before sending them.

Solution

1. Step 1: Understand streaming behavior in LangChain

   Streaming means tokens are sent one by one as soon as they are generated, not waiting for the full response.

2. Step 2: Match streaming=True effect

   Setting streaming=True activates this immediate token sending behavior.

3. Final Answer:

   It sends tokens immediately as they are generated. -> Option A

4. Quick Check:

   Streaming = immediate token sending [OK]

Hint: Streaming means tokens flow out live, not delayed [OK]

Common Mistakes:

• Thinking streaming buffers all tokens first
• Confusing streaming with disabling callbacks
• Assuming streaming delays output

2. Which of the following is the correct way to enable streaming with callbacks in LangChain?

easy

A. llm = OpenAI(streaming=True, callbacks=[MyCallbackHandler()])

B. llm = OpenAI(streaming=False, callbacks=MyCallbackHandler)

C. llm = OpenAI(callbacks=True, streaming=[MyCallbackHandler()])

D. llm = OpenAI(stream=True, callback=[MyCallbackHandler()])

Solution

1. Step 1: Recall correct parameter names

   LangChain's OpenAI class uses 'streaming=True' and 'callbacks' as a list of handlers.

2. Step 2: Check each option's syntax

   llm = OpenAI(streaming=True, callbacks=[MyCallbackHandler()]) correctly uses streaming=True and callbacks as a list. Others misuse parameter names or types.

3. Final Answer:

   llm = OpenAI(streaming=True, callbacks=[MyCallbackHandler()]) -> Option A

4. Quick Check:

   Correct params: streaming=True, callbacks=[handler] [OK]

Hint: Use streaming=True and callbacks as a list [OK]

Common Mistakes:

• Using streaming=False to try enabling streaming
• Passing callbacks as a single object, not a list
• Misspelling parameter names like 'stream' or 'callback'

3. Given this code snippet:

from langchain.callbacks.base import BaseCallbackHandler

class PrintTokens(BaseCallbackHandler):
    def on_llm_new_token(self, token: str, **kwargs):
        print(token, end='')

llm = OpenAI(streaming=True, callbacks=[PrintTokens()])
llm('Hello world')

What will be the output behavior?

medium

A. Prints 'Hello world' all at once after generation completes.

B. Raises a syntax error due to missing imports.

C. Prints nothing because callbacks are not supported.

D. Prints each token of 'Hello world' immediately as it is generated.

Solution

1. Step 1: Understand the callback handler

   The PrintTokens class prints each token immediately when on_llm_new_token is called.

2. Step 2: Streaming enabled triggers token callbacks live

   With streaming=True, tokens are sent and printed one by one as generated.

3. Final Answer:

   Prints each token of 'Hello world' immediately as it is generated. -> Option D

4. Quick Check:

   Streaming + on_llm_new_token = live token print [OK]

Hint: Streaming with on_llm_new_token prints tokens live [OK]

Common Mistakes:

• Expecting full output after completion
• Assuming callbacks don't work with streaming
• Missing that print uses end='' to avoid newlines

4. What is the main issue with this code snippet for streaming in LangChain?

llm = OpenAI(streaming=True, callbacks=PrintTokens())
llm('Test')

medium

A. The PrintTokens class is missing required methods.

B. streaming=True is not a valid parameter for OpenAI.

C. Callbacks must be passed as a list, not a single instance.

D. The llm call should be awaited with async syntax.

Solution

1. Step 1: Check callback parameter type

   LangChain expects callbacks as a list, even if only one handler is used.

2. Step 2: Identify error cause

   Passing callbacks=PrintTokens() (not in a list) causes a type error or unexpected behavior.

3. Final Answer:

   Callbacks must be passed as a list, not a single instance. -> Option C

4. Quick Check:

   Callbacks = list of handlers [OK]

Hint: Always wrap callbacks in a list, even if one [OK]

Common Mistakes:

• Passing a single callback object directly
• Assuming streaming=True is invalid
• Forgetting to implement callback methods

5. You want to build a chatbot that shows user responses token-by-token as they are generated. Which combination of LangChain features should you use in production?

hard

A. Use streaming=True with callbacks, but disable token printing to improve speed.

B. Use streaming=True with a callback handler implementing on_llm_new_token to display tokens live.

C. Use streaming=True but no callbacks, then print the final output after completion.

D. Use streaming=False and collect all tokens before displaying the full response.

Solution

1. Step 1: Identify streaming usage for live token display

   Streaming must be enabled to get tokens as they generate, not after full response.

2. Step 2: Use callback handler to process tokens live

   Implementing on_llm_new_token in a callback lets you display tokens immediately.

3. Step 3: Confirm best practice for production chatbot

   Combining streaming=True with a callback that prints tokens live is the correct approach.

4. Final Answer:

   Use streaming=True with a callback handler implementing on_llm_new_token to display tokens live. -> Option B

5. Quick Check:

   Streaming + on_llm_new_token = live chatbot tokens [OK]

Hint: Streaming plus on_llm_new_token callback shows tokens live [OK]

Common Mistakes:

• Disabling streaming and expecting live tokens
• Not using callbacks to handle tokens
• Printing tokens only after full response