Cost tracking across runs in LangChain - Step-by-Step Execution

Concept Flow - Cost tracking across runs

Start Run

↓

Initialize Cost Tracker

↓

Execute LangChain Run

↓

Calculate API Call Costs

↓

Add Costs to Tracker

↓

Save/Update Total Cost

↓

End Run

↓

Retrieve Total Cost for Reporting

This flow shows how cost tracking starts before a LangChain run, accumulates API call costs during execution, updates totals, and ends with a report.

Execution Sample

LangChain

from langchain.callbacks import get_openai_callback

with get_openai_callback() as cb:
    result = chain.run(input)
    print(cb.total_cost)

This code runs a LangChain chain while tracking the total API cost used during that run.

Execution Table

Step	Action	API Calls Made	Cost This Step (USD)	Total Cost So Far (USD)	Output
1	Start run and initialize cost tracker	0	0.00	0.00	Cost tracker ready
2	Run chain with input	2	0.004	0.004	Intermediate results generated
3	Run additional chain step	1	0.002	0.006	Final result produced
4	Print total cost	0	0.000	0.006	0.006 printed
5	End run	0	0.000	0.006	Run complete

💡 Run ends after printing total cost; total API cost tracked is 0.006 USD

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
total_cost	0.00	0.004	0.006	0.006	0.006
api_calls	0	2	3	3	3

Key Moments - 2 Insights

Why does total_cost increase only after API calls?

Because cost is calculated based on API usage during chain execution, as shown in steps 2 and 3 where API calls add cost.

Does printing the cost affect the total cost?

No, printing is just output and does not make API calls, so total_cost stays the same as seen in step 4.

Visual Quiz - 3 Questions

Look at the execution table, what is the total_cost after step 3?

A 0.006

B 0.004

C 0.002

D 0.000

💡 Hint

Check the 'Total Cost So Far (USD)' column at step 3 in the execution table.

At which step does the first API call cost get added?

A Step 1

B Step 4

C Step 2

D Step 5

💡 Hint

Look at the 'Cost This Step (USD)' column to see when cost first appears.

If the chain made one more API call at step 3, how would total_cost change?

A Decrease by 0.002 USD

B Increase by about 0.002 USD

C Stay the same

D Reset to zero

💡 Hint

Refer to cost increments per API call in the execution table rows 2 and 3.

Concept Snapshot

Use get_openai_callback() as a context manager to track API costs during LangChain runs.
Inside the with block, run your chain.
Access cb.total_cost to get the cost for that run.
Costs accumulate only when API calls happen.
Print or save total_cost after run to monitor expenses.

Full Transcript

This visual execution shows how LangChain tracks API costs across runs. First, a cost tracker is initialized before the chain runs. Each API call during the chain adds to the total cost. After the run, the total cost is printed. Variables like total_cost and api_calls update step-by-step. Key moments clarify that costs only increase with API calls and printing does not affect cost. The quiz tests understanding of cost accumulation and tracking steps.

Practice

1. What is the main purpose of using get_openai_callback() in LangChain?

easy

A. To connect LangChain with external databases

B. To speed up the execution of LangChain chains

C. To store the output of LangChain chains permanently

D. To track token usage and cost during LangChain runs

Solution

1. Step 1: Understand the role of get_openai_callback()

   This function is designed to monitor token usage and cost when running LangChain chains.

2. Step 2: Compare with other options

   The other options describe unrelated functionalities like speeding up execution, storing outputs, or database connections, which get_openai_callback() does not do.

3. Final Answer:

   To track token usage and cost during LangChain runs -> Option D

4. Quick Check:

   Cost tracking = A [OK]

Hint: Remember: callback tracks usage and cost only [OK]

Common Mistakes:

• Thinking it speeds up chain execution
• Confusing it with output storage
• Assuming it manages database connections

2. Which of the following is the correct way to use get_openai_callback() to track cost across multiple LangChain calls?

easy

A. with get_openai_callback() as cb:
chain.run(input1)
chain.run(input2)
print(cb.total_cost)

B. cb = get_openai_callback()
chain.run(input1)
chain.run(input2)
print(cb.total_cost)

C. chain.run(input1)
chain.run(input2)
cb = get_openai_callback()
print(cb.total_cost)

D. print(get_openai_callback().total_cost)
chain.run(input1)
chain.run(input2)

Solution

1. Step 1: Identify correct usage pattern

   The get_openai_callback() must be used as a context manager with a with block to track usage across multiple calls.

2. Step 2: Analyze options

   with get_openai_callback() as cb: chain.run(input1) chain.run(input2) print(cb.total_cost) correctly wraps calls inside the with block and accesses cb.total_cost after. The other options misuse the callback or access cost before running chains.

3. Final Answer:

   with get_openai_callback() as cb: chain.run(input1) chain.run(input2) print(cb.total_cost) -> Option A

4. Quick Check:

   Use with block for callback [OK]

Hint: Always use with to track cost across runs [OK]

Common Mistakes:

• Not using with block for callback
• Accessing cost before running chains
• Creating callback after chain runs

3. Given the code below, what will be printed?

with get_openai_callback() as cb:
    chain.run("Hello")
    chain.run("World")
print(cb.total_tokens)

medium

A. Zero, because tokens are not counted automatically

B. The total number of tokens used in both runs combined

C. The number of tokens used only in the last run

D. An error because total_tokens is not a valid attribute

Solution

1. Step 1: Understand token counting in callback

   Inside the with block, all calls to chain.run() accumulate token usage tracked by cb.

2. Step 2: Check what cb.total_tokens represents

   This attribute holds the total tokens used during the entire with block, so it sums tokens from both runs.

3. Final Answer:

   The total number of tokens used in both runs combined -> Option B

4. Quick Check:

   Total tokens = sum of all runs [OK]

Hint: Tokens add up inside the with block [OK]

Common Mistakes:

• Thinking it counts only last run tokens
• Assuming tokens are not tracked automatically
• Believing total_tokens is invalid

4. Identify the error in the following code snippet for tracking cost:

cb = get_openai_callback()
chain.run("Test")
print(cb.total_cost)

medium

A. The chain.run() method must be called before creating callback

B. The total_cost attribute does not exist on callback

C. Callback is not used as a context manager, so cost is not tracked

D. The print statement should be inside the callback block

Solution

1. Step 1: Check callback usage

   The callback must be used inside a with block to track usage properly. Here, it is created but not used as a context manager.

2. Step 2: Understand consequences

   Without the with block, the callback does not track token or cost usage, so total_cost will not reflect actual usage.

3. Final Answer:

   Callback is not used as a context manager, so cost is not tracked -> Option C

4. Quick Check:

   Use with for tracking [OK]

Hint: Always use with to activate callback tracking [OK]

Common Mistakes:

• Creating callback without with
• Assuming attributes exist without context
• Placing print outside tracking scope

5. You want to track token usage and cost for multiple LangChain runs, but also reset tracking between different user sessions. Which approach correctly achieves this?

hard

A. Use separate with get_openai_callback() as cb: blocks for each session

B. Create one callback outside all sessions and reuse it without resetting

C. Call get_openai_callback() once and manually reset cb.total_cost to zero

D. Track cost by printing cb.total_cost after each run without context manager

Solution

1. Step 1: Understand session-based tracking needs

   Each user session should have isolated cost tracking to avoid mixing token counts and costs.

2. Step 2: Evaluate approaches

   Using separate with blocks creates fresh callbacks per session, resetting counts automatically. Other options either reuse callbacks incorrectly or try manual resets which are unsupported.

3. Final Answer:

   Use separate with get_openai_callback() as cb: blocks for each session -> Option A

4. Quick Check:

   Separate with blocks reset tracking [OK]

Hint: Use new with block per session to reset cost [OK]

Common Mistakes:

• Reusing one callback for all sessions
• Trying to manually reset callback attributes
• Not using with blocks for tracking