Setting up LangSmith tracing in LangChain - Visual Walkthrough

Concept Flow - Setting up LangSmith tracing

Import LangSmithTracer

↓

Create tracer instance

↓

Configure LangChain to use tracer

↓

Run LangChain operations

↓

Tracer collects and sends data

↓

View traces in LangSmith UI

This flow shows how to import, create, and configure LangSmith tracing, then run LangChain code that sends trace data to LangSmith.

Execution Sample

LangChain

from langchain import OpenAI, LLMChain
from langchain_experimental.langsmith import LangSmithTracer

tracer = LangSmithTracer()
llm = OpenAI()
llm.chain_type = "stuff"
llm.tracer = tracer

chain = LLMChain(llm=llm, prompt=some_prompt)
result = chain.run("Hello")

This code sets up LangSmithTracer, attaches it to an OpenAI LLM, then runs a chain to generate output while tracing.

Execution Table

Step	Action	Evaluation	Result
1	Import LangSmithTracer	Module found	LangSmithTracer class ready
2	Create tracer instance	tracer = LangSmithTracer()	tracer object created
3	Create OpenAI LLM instance	llm = OpenAI()	llm object created
4	Attach tracer to llm	llm.tracer = tracer	llm now sends traces
5	Create LLMChain with llm	chain = LLMChain(llm=llm, prompt=some_prompt)	chain ready to run
6	Run chain with input 'Hello'	chain.run('Hello')	Output generated, trace sent
7	Tracer collects data	Trace data captured	Data sent to LangSmith UI
8	View traces	Open LangSmith UI	Trace visible for analysis

💡 Tracing setup complete and data sent after chain run

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6	Final
tracer	undefined	LangSmithTracer instance	LangSmithTracer instance	LangSmithTracer instance	LangSmithTracer instance	LangSmithTracer instance	LangSmithTracer instance
llm	undefined	undefined	OpenAI instance	OpenAI instance with tracer	OpenAI instance with tracer	OpenAI instance with tracer	OpenAI instance with tracer
chain	undefined	undefined	undefined	undefined	LLMChain instance	LLMChain instance	LLMChain instance
result	undefined	undefined	undefined	undefined	undefined	Output string	Output string

Key Moments - 2 Insights

Why do we attach the tracer to the LLM instead of the chain?

The tracer is attached to the LLM because it captures the calls and responses of the language model itself. The chain uses the LLM, so tracing the LLM covers the core operations. This is shown in execution_table step 4 where llm.tracer is set.

What happens if we run the chain without setting the tracer?

If the tracer is not set on the LLM, no trace data is collected or sent. The chain still runs and produces output, but tracing is inactive. This is implied by the absence of tracer in variable_tracker before step 4.

Visual Quiz - 3 Questions

Look at the execution_table at step 6, what happens when chain.run('Hello') is called?

A The OpenAI LLM is instantiated

B The tracer is created

C The chain generates output and sends trace data

D The prompt is defined

💡 Hint

Refer to execution_table row with Step 6 describing chain.run action and result

According to variable_tracker, when does the 'tracer' variable get its instance value?

A After Step 2

B After Step 3

C After Step 5

D After Step 6

💡 Hint

Check variable_tracker row for 'tracer' and see when it changes from undefined

If we did not assign llm.tracer = tracer, what would change in the execution?

A The chain would fail to run

B Trace data would not be collected during chain.run

C The tracer would not be created

D The prompt would not be set

💡 Hint

Look at key_moments explaining the importance of attaching tracer to llm

Concept Snapshot

Setting up LangSmith tracing:
1. Import LangSmithTracer
2. Create tracer instance
3. Attach tracer to your LLM (e.g., OpenAI)
4. Run your LangChain code
5. Tracer collects and sends trace data
6. View traces in LangSmith UI

Full Transcript

To set up LangSmith tracing, first import the LangSmithTracer class. Then create an instance of it. Next, create your language model object, such as OpenAI. Attach the tracer instance to the llm by setting llm.tracer = tracer. After this, create your LangChain chain using the llm. When you run the chain with input, the tracer collects data about the calls and responses. This data is sent to the LangSmith UI where you can view detailed traces of your language model usage. Without attaching the tracer to the llm, no trace data is collected even if the chain runs successfully.

Practice

1. What is the main purpose of setting up LangSmith tracing in a LangChain application?

easy

A. To encrypt data passed between LangChain components

B. To speed up the execution of LangChain components

C. To automatically fix errors in your LangChain code

D. To monitor and visualize the steps of your LangChain workflows

Solution

1. Step 1: Understand LangSmith tracing purpose

   LangSmith tracing is designed to help watch and understand the steps your LangChain app takes.

2. Step 2: Identify the correct purpose

   It does not speed up execution, fix errors automatically, or encrypt data, but helps monitor and visualize workflows.

3. Final Answer:

   To monitor and visualize the steps of your LangChain workflows -> Option D

4. Quick Check:

   Tracing = Monitoring steps [OK]

Hint: Tracing means watching steps clearly in LangChain [OK]

Common Mistakes:

• Thinking tracing speeds up code
• Assuming tracing fixes bugs automatically
• Confusing tracing with data encryption

2. Which of the following is the correct way to create a LangChainTracer for LangSmith tracing?

easy

A. tracer = LangChainTracer()

B. tracer = createTracer()

C. tracer = new LangSmithTracer()

D. tracer = LangSmith.createTracer()

Solution

1. Step 1: Recall LangChainTracer creation syntax

   The LangChainTracer is created by calling its constructor directly: LangChainTracer()

2. Step 2: Check options for correct syntax

   Options B, C, and D use incorrect function or class names or syntax not used in LangChain.

3. Final Answer:

   tracer = LangChainTracer() -> Option A

4. Quick Check:

   Constructor call = LangChainTracer() [OK]

Hint: Use LangChainTracer() constructor to create tracer [OK]

Common Mistakes:

• Using wrong class names like LangSmithTracer
• Calling non-existent functions like createTracer()
• Using 'new' keyword which is not Python syntax

3. Given the code snippet below, what will be the effect of passing the tracer to the LLM?

from langchain.chat_models import ChatOpenAI
from langchain.callbacks import LangChainTracer

tracer = LangChainTracer()
llm = ChatOpenAI(callbacks=[tracer])
response = llm.chat([{'role': 'user', 'content': 'Hello!'}])

medium

A. The LLM will log its steps to LangSmith for tracing

B. The LLM will run without any tracing or logging

C. The code will raise a syntax error due to wrong callback usage

D. The LLM will ignore the tracer and produce no output

Solution

1. Step 1: Understand passing tracer as callback

   Passing LangChainTracer in callbacks enables tracing of LLM steps.

2. Step 2: Analyze code behavior

   The LLM will send its internal steps to LangSmith via the tracer, enabling monitoring.

3. Final Answer:

   The LLM will log its steps to LangSmith for tracing -> Option A

4. Quick Check:

   Callbacks with tracer = tracing enabled [OK]

Hint: Callbacks=[tracer] enables LangSmith tracing [OK]

Common Mistakes:

• Assuming no tracing happens without explicit start call
• Thinking callbacks cause syntax errors here
• Believing tracer disables output

4. Identify the error in this LangSmith tracing setup code:

from langchain.chat_models import ChatOpenAI
from langchain.callbacks import LangChainTracer

tracer = LangChainTracer
llm = ChatOpenAI(callbacks=[tracer])

medium

A. LangChainTracer is not imported correctly

B. LangChainTracer is assigned without parentheses, missing instantiation

C. Callbacks list should be empty for tracing

D. ChatOpenAI does not accept callbacks parameter

Solution

1. Step 1: Check LangChainTracer assignment

   tracer = LangChainTracer misses parentheses, so tracer is a class, not an instance.

2. Step 2: Analyze usage in callbacks

   Passing callbacks=[tracer] passes the class instead of an instance, causing a runtime error when callbacks are used.

3. Final Answer:

   LangChainTracer is assigned without parentheses, missing instantiation -> Option B

4. Quick Check:

   Instantiate with () to create tracer object [OK]

Hint: Always instantiate classes with () before use [OK]

Common Mistakes:

• Assigning class instead of instance
• Calling instance as function
• Ignoring callbacks parameter usage

5. You want to trace both an LLM and a chain in LangChain using LangSmith. Which setup correctly enables tracing for both components?

hard

A. llm = ChatOpenAI()
chain = SomeChain(llm=llm)
tracer = LangChainTracer()
tracer.start()

B. tracer = LangChainTracer
llm = ChatOpenAI(callbacks=tracer)
chain = SomeChain(llm=llm, callbacks=tracer)

C. tracer = LangChainTracer()
llm = ChatOpenAI(callbacks=[tracer])
chain = SomeChain(llm=llm, callbacks=[tracer])

D. tracer = LangChainTracer()
llm = ChatOpenAI()
chain = SomeChain(llm=llm)

Solution

1. Step 1: Instantiate LangChainTracer correctly

   Use tracer = LangChainTracer() to create the tracer instance.

2. Step 2: Pass tracer in callbacks for both LLM and chain

   Both components accept callbacks as a list; passing [tracer] enables tracing on both.

3. Step 3: Evaluate options

   tracer = LangChainTracer() llm = ChatOpenAI(callbacks=[tracer]) chain = SomeChain(llm=llm, callbacks=[tracer]) correctly instantiates tracer and passes it as a list to both components. Others miss instantiation, use wrong types, or omit callbacks.

4. Final Answer:

   tracer = LangChainTracer() llm = ChatOpenAI(callbacks=[tracer]) chain = SomeChain(llm=llm, callbacks=[tracer]) -> Option C

5. Quick Check:

   Instantiate tracer and pass as list to callbacks [OK]

Hint: Instantiate tracer and pass as list to all callbacks [OK]

Common Mistakes:

• Not instantiating tracer with ()
• Passing tracer directly instead of in a list
• Forgetting to add callbacks to chain