Viewing trace details and latency in LangChain - Mini Project: Build & Apply

Viewing trace details and latency in LangChain

📖 Scenario: You are building a simple LangChain application that calls an LLM to generate text. You want to see detailed trace information and latency for each step to understand how long each part takes.

🎯 Goal: Enable LangChain tracing to view detailed trace logs and latency information for your chain calls.

📋 What You'll Learn

Create a LangChain LLM chain with OpenAI

Enable tracing on the chain

Call the chain with a prompt

Access and print the trace details and latency

💡 Why This Matters

🌍 Real World

Developers use LangChain to build AI applications and need to monitor performance and costs by viewing trace details and latency.

💼 Career

Understanding how to trace and measure latency in LangChain is important for AI engineers and developers working with language models to optimize user experience and resource usage.

1

Set up the LangChain LLM chain

Import OpenAI and LLMChain from langchain. Create an llm variable as OpenAI(model_name='text-davinci-003'). Then create a simple LLMChain called chain with the llm and a prompt template 'Say hello to {name}'.

LangChain

# Import OpenAI and LLMChain
# Create llm = OpenAI(model_name='text-davinci-003')
# Create chain = LLMChain(llm=llm, prompt='Say hello to {name}')

Need a hint?

Use OpenAI from langchain.llms and LLMChain from langchain.chains. The prompt can be a simple string with {name} as a variable.

2

Enable tracing on the chain

Set the verbose parameter of the LLMChain to True to enable tracing. Update the chain creation line to include verbose=True.

LangChain

from langchain.llms import OpenAI
from langchain.chains import LLMChain

llm = OpenAI(model_name='text-davinci-003')
# Enable verbose tracing on the chain
chain = LLMChain(llm=llm, prompt='Say hello to {name}', verbose=True)

Need a hint?

Add verbose=True when creating the LLMChain to see trace details.

3

Call the chain with a prompt

Call the chain with name='Alice' and save the result in a variable called result. Use chain.run(name='Alice').

LangChain

from langchain.llms import OpenAI
from langchain.chains import LLMChain

llm = OpenAI(model_name='text-davinci-003')
chain = LLMChain(llm=llm, prompt='Say hello to {name}', verbose=True)

# Call the chain with name='Alice' and save to result
result = chain.run(name='Alice')

Need a hint?

Use chain.run(name='Alice') to get the output for the prompt.

4

Access and print trace details and latency

Import get_openai_callback from langchain.callbacks. Use it as a context manager around the chain.run(name='Alice') call to capture trace details. After the call, assign the callback object to a variable callback. Then access callback.total_tokens and callback.total_time to get token usage and latency.

LangChain

from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.callbacks import get_openai_callback

llm = OpenAI(model_name='text-davinci-003')
chain = LLMChain(llm=llm, prompt='Say hello to {name}', verbose=True)

# Use get_openai_callback to capture trace details and latency
with get_openai_callback() as callback:
    result = chain.run(name='Alice')

# Access callback.total_tokens and callback.total_time below

Need a hint?

Use get_openai_callback as a context manager to capture usage and latency info.