Question reformulation with history in LangChain - Performance & Optimization

Performance: Question reformulation with history

MEDIUM IMPACT

This concept affects the speed and responsiveness of conversational AI by managing how previous user inputs are reused and reformulated for new queries.

Reformulating user questions using conversation history in a chatbot

LangChain

def reformulate_question(history, new_question, max_history=3):
    recent_context = ' '.join(history[-max_history:])
    return f"{recent_context} {new_question}"

Limits context to recent turns, reducing processing time and memory use, improving response speed.

📈 Performance Gainreduces processing time by 70% on long histories; lowers memory footprint

Reformulating user questions using conversation history in a chatbot

LangChain

def reformulate_question(history, new_question):
    full_context = ''
    for turn in history:
        full_context += turn + ' '
    full_context += new_question
    return full_context

Concatenating entire history every time causes repeated processing and large context size, slowing response and increasing memory use.

📉 Performance Costblocks rendering for 100+ ms on long histories; increases memory usage linearly with history length

Performance Comparison

Pattern	Context Size	Processing Time	Memory Use	Verdict
Full history concatenation	Large (all turns)	High (linear growth)	High (linear growth)	[X] Bad
Limited recent history	Small (last 3 turns)	Low (constant)	Low (constant)	[OK] Good

Rendering Pipeline

The reformulation process impacts the interaction responsiveness stage by controlling how much data is processed before generating a response.

→Input Processing

→Computation

→Response Generation

⚠️ BottleneckComputation stage due to large context concatenation and tokenization

Core Web Vital Affected

INP

This concept affects the speed and responsiveness of conversational AI by managing how previous user inputs are reused and reformulated for new queries.

Optimization Tips

1Avoid concatenating the entire conversation history for each reformulation.

2Limit history context to recent turns to reduce processing overhead.

3Cache or reuse reformulated questions when possible to save computation.

Performance Quiz - 3 Questions

What is the main performance issue when reformulating questions using the entire conversation history every time?

A Increased processing time and memory use due to large context size

B Reduced accuracy of the reformulated question

C Slower network requests to the server

D More user input errors

DevTools: Performance

How to check: Record a performance profile while interacting with the chatbot; look for long scripting tasks during question reformulation.

What to look for: High CPU time or long scripting tasks indicate inefficient history processing; shorter tasks show optimized reformulation.