Few-shot prompt templates in LangChain - Performance & Optimization

Performance: Few-shot prompt templates

MEDIUM IMPACT

This concept affects the speed of generating AI responses by controlling prompt size and complexity, impacting initial load and interaction responsiveness.

Creating a prompt template with examples to guide AI responses

LangChain

from langchain.prompts import PromptTemplate, FewShotPromptTemplate

example_prompt = PromptTemplate(
    input_variables=["input", "output"],
    template="Question: {input}\nAnswer: {output}\n"
)

examples = [
  {"input": "What is AI?", "output": "AI is artificial intelligence."},
  {"input": "Define ML.", "output": "ML is machine learning."}
]

prompt = FewShotPromptTemplate(
  examples=examples,
  example_prompt=example_prompt,
  prefix="Answer the following questions:\n",
  suffix="Question: {input}",
  input_variables=["input"]
)

Reducing examples lowers token count, speeding up prompt processing and response time.

📈 Performance GainCuts token count roughly in half, reducing response latency by 150-300ms

Creating a prompt template with examples to guide AI responses

LangChain

from langchain.prompts import PromptTemplate, FewShotPromptTemplate

example_prompt = PromptTemplate(
    input_variables=["input", "output"],
    template="Question: {input}\nAnswer: {output}\n"
)

examples = [
  {"input": "What is AI?", "output": "AI is artificial intelligence."},
  {"input": "Define ML.", "output": "ML is machine learning."},
  {"input": "Explain NLP.", "output": "NLP is natural language processing."},
  {"input": "What is a neural network?", "output": "A neural network is a model inspired by the brain."}
]

prompt = FewShotPromptTemplate(
  examples=examples,
  example_prompt=example_prompt,
  prefix="Answer the following questions:\n",
  suffix="Question: {input}",
  input_variables=["input"]
)

Using too many examples increases prompt size, causing slower token processing and higher latency.

📉 Performance CostIncreases token count by 4 examples, adding ~200 tokens, which can block response generation for 200-400ms

Performance Comparison

Pattern	Token Count	Inference Time	Latency Impact	Verdict
Many examples (4+)	High (~400 tokens)	Longer (~400ms)	Higher latency	[X] Bad
Few examples (2)	Medium (~200 tokens)	Shorter (~200ms)	Lower latency	[OK] Good

Rendering Pipeline

Few-shot prompt templates increase the input size sent to the AI model, affecting tokenization and model inference time, which impacts interaction responsiveness.

→Tokenization

→Model Inference

→Network Transfer

⚠️ BottleneckModel Inference due to larger input tokens

Core Web Vital Affected

INP

This concept affects the speed of generating AI responses by controlling prompt size and complexity, impacting initial load and interaction responsiveness.

Optimization Tips

1Keep few-shot examples short and minimal to reduce token count.

2Avoid adding unnecessary examples that increase prompt size.

3Test prompt size impact on response latency using DevTools Network and Performance panels.

Performance Quiz - 3 Questions

How does adding more examples in a few-shot prompt template affect AI response time?

A It increases token count, causing slower response times

B It decreases token count, speeding up responses

C It has no effect on response time

D It reduces network latency

DevTools: Network and Performance panels

How to check: Record a performance profile while sending prompts; check request payload size and response time in Network panel; analyze CPU time in Performance panel.

What to look for: Look for large request payloads (prompt size) and long model response times indicating slow inference.

Practice

1. What is the main purpose of a few-shot prompt template in Langchain?

easy

A. To provide example prompts and responses to guide AI behavior

B. To store large datasets for training AI models

C. To execute code on the AI server

D. To create user interfaces for AI applications

Solution

1. Step 1: Understand few-shot prompt templates

   Few-shot prompt templates include example prompts and responses to teach AI how to answer.

2. Step 2: Identify the main purpose

   The main goal is to guide AI behavior by showing examples, not to store data or create interfaces.

3. Final Answer:

   To provide example prompts and responses to guide AI behavior -> Option A

4. Quick Check:

   Few-shot prompt templates guide AI with examples = A [OK]

Hint: Remember: few-shot means showing examples to teach AI [OK]

Common Mistakes:

• Confusing prompt templates with data storage
• Thinking they run code instead of guiding AI
• Assuming they build UI components

2. Which of the following is the correct way to create a few-shot prompt template in Langchain?

easy

A. FewShotPromptTemplate(data=examples, prompt=example_prompt, suffix=prefix_text)

B. FewShotPromptTemplate(samples=examples, prompt_template=example_prompt, header=prefix_text)

C. FewShotPromptTemplate(examples=examples, example_prompt=example_prompt, prefix=prefix_text)

D. FewShotPromptTemplate(inputs=examples, prompt=example_prompt, footer=prefix_text)

Solution

1. Step 1: Recall Langchain few-shot prompt syntax

   The correct constructor uses parameters: examples, example_prompt, and prefix.

2. Step 2: Match parameters to options

   Only FewShotPromptTemplate(examples=examples, example_prompt=example_prompt, prefix=prefix_text) uses the exact parameter names required by Langchain's FewShotPromptTemplate.

3. Final Answer:

   FewShotPromptTemplate(examples=examples, example_prompt=example_prompt, prefix=prefix_text) -> Option C

4. Quick Check:

   Correct parameter names = B [OK]

Hint: Check parameter names exactly as in Langchain docs [OK]

Common Mistakes:

• Using wrong parameter names like data or samples
• Mixing prefix with suffix or footer
• Confusing example_prompt with prompt_template

3. Given this code snippet, what will be the output of print(prompt_template.format(input="Translate to French: Hello") )?

examples = [{"input": "Hello", "output": "Bonjour"}]
example_prompt = PromptTemplate(input_variables=["input", "output"], template="Input: {input}\nOutput: {output}")
prompt_template = FewShotPromptTemplate(examples=examples, example_prompt=example_prompt, prefix="Translate English to French:\n", suffix="\nInput: {input}\nOutput:", input_variables=["input"])

medium

A. Translate English to French:
Input: Translate to French: Hello
Output:

B. Translate English to French:
Input: Hello
Output: Bonjour
Translate to French: Hello

C. Input: Hello
Output: Bonjour
Translate English to French: Hello

D. Translate English to French:
Input: Hello
Output: Bonjour
Input: Translate to French: Hello
Output:

Solution

1. Step 1: Understand few-shot prompt formatting

   The prompt includes the prefix, then example prompts formatted with example data, then the new input prompt.

2. Step 2: Apply formatting to given input

   The prefix is "Translate English to French:", then example "Input: Hello\nOutput: Bonjour", then the new input "Input: Translate to French: Hello\nOutput:" (empty output to be filled by AI).

3. Final Answer:

   Translate English to French: Input: Hello Output: Bonjour Input: Translate to French: Hello Output: -> Option D

4. Quick Check:

   Prefix + example + new input prompt = C [OK]

Hint: Few-shot templates show prefix, examples, then new input [OK]

Common Mistakes:

• Ignoring prefix text in output
• Not formatting new input as 'Input: ... Output:'
• Confusing example data with new input

4. What is the error in this code snippet that tries to create a few-shot prompt template?

examples = [{"input": "Hi", "output": "Salut"}]
example_prompt = PromptTemplate(input_variables=["input", "output"], template="Input: {input}\nOutput: {output}")
prompt_template = FewShotPromptTemplate(examples=examples, example_prompt=example_prompt, prefix="Translate:")
print(prompt_template.format(input="Hello"))

medium

A. PromptTemplate cannot have input_variables

B. Missing input_variables parameter in FewShotPromptTemplate constructor

C. Prefix must be a function, not a string

D. examples list should be empty for FewShotPromptTemplate

Solution

1. Step 1: Check FewShotPromptTemplate required parameters

   FewShotPromptTemplate requires input_variables parameter to know which inputs to expect.

2. Step 2: Identify missing parameter

   The code misses input_variables in FewShotPromptTemplate, causing an error when calling format.

3. Final Answer:

   Missing input_variables parameter in FewShotPromptTemplate constructor -> Option B

4. Quick Check:

   input_variables missing = D [OK]

Hint: Always include input_variables when creating prompt templates [OK]

Common Mistakes:

• Omitting input_variables in FewShotPromptTemplate
• Thinking prefix must be a function
• Assuming examples can be empty

5. You want to create a few-shot prompt template that filters out examples with empty outputs before formatting. Which approach correctly applies this filtering in Langchain?

hard

A. Filter the examples list before passing it to FewShotPromptTemplate constructor

B. Use a custom example_prompt that skips empty outputs during formatting

C. Set prefix to None and rely on Langchain to ignore empty outputs

D. Pass all examples and filter outputs after calling prompt_template.format()

Solution

1. Step 1: Understand filtering in few-shot templates

   FewShotPromptTemplate uses the examples list as-is; filtering must happen before passing examples.

2. Step 2: Evaluate options for filtering

   Only filtering the examples list before creating the template ensures empty outputs are excluded properly.

3. Final Answer:

   Filter the examples list before passing it to FewShotPromptTemplate constructor -> Option A

4. Quick Check:

   Pre-filter examples before constructor = A [OK]

Hint: Filter examples before creating the prompt template [OK]

Common Mistakes:

• Trying to filter inside example_prompt formatting
• Assuming Langchain auto-filters empty outputs
• Filtering after formatting instead of before