Why observability is essential for LLM apps in LangChain - See It in Action

Why Observability is Essential for LLM Apps

📖 Scenario: You are building a simple LangChain app that uses a large language model (LLM) to answer questions. To make sure your app works well and you can fix problems quickly, you want to add observability features. Observability means you can see what is happening inside your app, like tracking inputs, outputs, and errors.

🎯 Goal: Build a basic LangChain app with observability by setting up the data, adding configuration for logging, implementing the core logic to run the LLM with logging, and completing the app with error handling and final logging setup.

📋 What You'll Learn

Create a LangChain LLM instance with a fixed model name

Add a configuration variable to enable logging

Use LangChain's callback manager to log inputs and outputs

Add error handling to log exceptions

💡 Why This Matters

🌍 Real World

Observability helps developers understand how their LLM apps behave in real time. It shows what inputs the model receives, what outputs it produces, and if any errors happen. This is like having a dashboard for your app's health.

💼 Career

Many companies use LLMs in production. Knowing how to add observability is important for maintaining, debugging, and improving these apps. It is a key skill for AI engineers and developers working with LangChain or similar frameworks.

1

Set up the LLM instance

Create a LangChain LLM instance called llm using OpenAI with the model name "gpt-3.5-turbo".

LangChain

# Your code here

Need a hint?

Use ChatOpenAI from langchain.chat_models and set model_name to "gpt-3.5-turbo".

2

Add logging configuration

Create a boolean variable called enable_logging and set it to True to enable observability logging.

LangChain

from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-3.5-turbo")

# Your code here

Need a hint?

Just create a variable enable_logging and set it to True.

3

Implement LLM call with logging

Import CallbackManager and StdOutCallbackHandler from langchain.callbacks. Create a callback_manager that uses StdOutCallbackHandler() only if enable_logging is True. Then create a new llm_with_logging instance of ChatOpenAI with the same model name and the callback_manager. Finally, call llm_with_logging with the prompt "What is observability?" and assign the result to response.

LangChain

from langchain.chat_models import ChatOpenAI
from langchain.callbacks import CallbackManager, StdOutCallbackHandler

llm = ChatOpenAI(model_name="gpt-3.5-turbo")
enable_logging = True

# Your code here

Need a hint?

Use CallbackManager with StdOutCallbackHandler() only if enable_logging is True. Pass it to the new LLM instance.

4

Add error handling and finalize observability

Wrap the LLM call in a try block. If an exception occurs, catch it with except Exception as e and print "Error:" followed by the exception e. This completes the observability setup by logging errors.

LangChain

from langchain.chat_models import ChatOpenAI
from langchain.callbacks import CallbackManager, StdOutCallbackHandler

llm = ChatOpenAI(model_name="gpt-3.5-turbo")
enable_logging = True

callback_manager = CallbackManager([StdOutCallbackHandler()]) if enable_logging else None

llm_with_logging = ChatOpenAI(model_name="gpt-3.5-turbo", callback_manager=callback_manager)

# Your code here

Need a hint?

Use a try block around the LLM call and catch exceptions to print errors.