Cost tracking across runs in LangChain

Introduction

Cost tracking helps you see how much you spend using LangChain over time. It keeps your spending clear and easy to manage.

You want to know how much each LangChain run costs to avoid surprises.

You need to compare costs between different runs or projects.

You want to track spending to stay within a budget.

You want to analyze which parts of your app use the most resources.

You want to log costs for reporting or billing purposes.

Syntax

LangChain

from langchain.callbacks import get_openai_callback

with get_openai_callback() as cb:
    # your LangChain code here
    result = chain.run(input)

print(f"Total Tokens: {cb.total_tokens}")
print(f"Prompt Tokens: {cb.prompt_tokens}")
print(f"Completion Tokens: {cb.completion_tokens}")
print(f"Total Cost (USD): ${cb.total_cost:.6f}")

The get_openai_callback() function tracks tokens and cost during the code inside the with block.

After the block, you can access token counts and cost from the callback object.

Examples

Tracks cost for a single chain run with input "Hello".

LangChain

from langchain.callbacks import get_openai_callback

with get_openai_callback() as cb:
    response = chain.run("Hello")

print(f"Cost: ${cb.total_cost:.6f}")

Tracks total cost for multiple runs inside one callback context.

LangChain

from langchain.callbacks import get_openai_callback

with get_openai_callback() as cb:
    for prompt in ["Hi", "How are you?"]:
        chain.run(prompt)

print(f"Total cost for all runs: ${cb.total_cost:.6f}")

Sample Program

This example runs the chain twice with different names. It tracks and prints the total tokens and cost used across both runs.

LangChain

from langchain.llms import OpenAI
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.callbacks import get_openai_callback

# Setup prompt and chain
prompt = PromptTemplate(template="Say hello to {name}", input_variables=["name"])
llm = OpenAI(temperature=0)
chain = LLMChain(llm=llm, prompt=prompt)

# Track cost across multiple runs
with get_openai_callback() as cb:
    print(chain.run(name="Alice"))
    print(chain.run(name="Bob"))

print(f"Total Tokens Used: {cb.total_tokens}")
print(f"Prompt Tokens: {cb.prompt_tokens}")
print(f"Completion Tokens: {cb.completion_tokens}")
print(f"Total Cost (USD): ${cb.total_cost:.6f}")

Important Notes

Costs depend on the model and token usage; check your OpenAI pricing for details.

Use get_openai_callback() to wrap all runs you want to track together.

Tracking cost helps you optimize your app and avoid unexpected charges.

Summary

Use get_openai_callback() to track tokens and cost during LangChain runs.

Wrap your code inside a with block to measure usage across multiple calls.

Access token counts and cost after the block to monitor spending.