Comparing prompt versions in LangChain - Step-by-Step Execution

Concept Flow - Comparing prompt versions

Define Prompt Version 1

↓

Define Prompt Version 2

↓

Input User Query

↓

Run Prompt Version 1 with Query

↓

Run Prompt Version 2 with Query

↓

Compare Outputs

↓

Decide Best Prompt Version

This flow shows how two prompt versions are defined, run with the same input, and their outputs compared to choose the best.

Execution Sample

LangChain

from langchain import PromptTemplate

prompt_v1 = PromptTemplate(template="Hello {name}!")
prompt_v2 = PromptTemplate(template="Hi there, {name}!")

input_data = {"name": "Alice"}

output_v1 = prompt_v1.format(**input_data)
output_v2 = prompt_v2.format(**input_data)

print(output_v1)
print(output_v2)

This code creates two prompt versions and formats them with the same input to compare their outputs.

Execution Table

Step	Action	Input	Output	Notes
1	Define prompt_v1	template="Hello {name}!"	PromptTemplate object created	Stores first prompt version
2	Define prompt_v2	template="Hi there, {name}!"	PromptTemplate object created	Stores second prompt version
3	Set input_data	{"name": "Alice"}	Dictionary with name key	Input for formatting
4	Format prompt_v1	input_data	"Hello Alice!"	First prompt output
5	Format prompt_v2	input_data	"Hi there, Alice!"	Second prompt output
6	Print output_v1	"Hello Alice!"	Displays: Hello Alice!	Output from prompt_v1
7	Print output_v2	"Hi there, Alice!"	Displays: Hi there, Alice!	Output from prompt_v2
8	Compare outputs	"Hello Alice!" vs "Hi there, Alice!"	Decide which sounds better	Comparison step
9	End	-	-	Comparison complete

💡 All steps executed to compare prompt outputs for same input.

Variable Tracker

Variable	Start	After Step 3	After Step 4	After Step 5	Final
prompt_v1	undefined	PromptTemplate(template="Hello {name}!")	Same	Same	Same
prompt_v2	undefined	PromptTemplate(template="Hi there, {name}!")	Same	Same	Same
input_data	undefined	{"name": "Alice"}	Same	Same	Same
output_v1	undefined	undefined	"Hello Alice!"	"Hello Alice!"	"Hello Alice!"
output_v2	undefined	undefined	undefined	"Hi there, Alice!"	"Hi there, Alice!"

Key Moments - 3 Insights

Why do both prompt versions use the same input_data?

Using the same input_data ensures a fair comparison of outputs, as shown in execution_table rows 3, 4, and 5.

What happens if the templates have different placeholders?

If placeholders differ, formatting will fail or produce incorrect output. Here, both use {name}, so formatting succeeds (rows 4 and 5).

How do we decide which prompt version is better?

We compare the outputs (row 8) to see which phrasing fits our needs or sounds better.

Visual Quiz - 3 Questions

Look at the execution_table, what is the output of prompt_v1 at step 4?

A "Hi there, Alice!"

B "Hello Alice!"

C "Hello {name}!"

D An error

💡 Hint

Check the Output column at step 4 in the execution_table.

At which step do we compare the outputs of both prompt versions?

A Step 5

B Step 3

C Step 8

D Step 7

💡 Hint

Look for the row mentioning 'Compare outputs' in the execution_table.

If input_data changed to {"name": "Bob"}, what would output_v2 be at step 5?

A "Hi there, Bob!"

B "Hi there, Alice!"

C "Hello Bob!"

D "Hello Alice!"

💡 Hint

Refer to variable_tracker and how formatting replaces {name} with input_data value.

Concept Snapshot

Comparing prompt versions:
- Define multiple PromptTemplate objects with different templates.
- Use the same input data to format each prompt.
- Run each prompt with input to get outputs.
- Compare outputs to choose the best prompt.
- Ensures fair testing of prompt wording.

Full Transcript

This visual execution shows how to compare two prompt versions in langchain. First, we create two PromptTemplate objects with different text templates. Then, we prepare the same input data dictionary with a name key. Next, we format each prompt with this input to produce output strings. We print both outputs to see their differences. Finally, we compare these outputs to decide which prompt version sounds better or fits our needs. Tracking variables shows prompt templates stay the same, input data is constant, and outputs change after formatting. Key moments clarify why same input is needed and how to handle placeholders. The quiz tests understanding of outputs and comparison steps.