Comparing prompt versions in LangChain - Mini Project: Build & Apply

Comparing prompt versions

📖 Scenario: You are building a simple tool to compare two versions of text prompts used in a language model application. This helps you see what changed between versions.

🎯 Goal: Create a Python script that stores two prompt versions, sets a threshold for similarity, compares the prompts, and outputs whether they are similar or different.

📋 What You'll Learn

Create two string variables named prompt_v1 and prompt_v2 with exact given texts

Create a float variable similarity_threshold set to 0.8

Write a function compare_prompts that takes two prompts and returns True if their similarity is above the threshold

Add a final line that calls compare_prompts with prompt_v1 and prompt_v2

💡 Why This Matters

🌍 Real World

Comparing prompt versions helps developers track changes and improvements in language model inputs.

💼 Career

Skills in text processing and comparison are useful for AI developers, data scientists, and software engineers working with natural language.

1

DATA SETUP: Define two prompt versions

Create two string variables called prompt_v1 and prompt_v2 with these exact values:
prompt_v1 = "What is the weather like today?"
prompt_v2 = "Can you tell me the weather forecast for today?"

LangChain

# Your code here

Hint

Use exact variable names and string values as shown.

2

CONFIGURATION: Set similarity threshold

Create a float variable called similarity_threshold and set it to 0.8

LangChain

prompt_v1 = "What is the weather like today?"
prompt_v2 = "Can you tell me the weather forecast for today?"
# Your code here

Hint

Use the exact variable name and value.

3

CORE LOGIC: Write a function to compare prompts

Write a function called compare_prompts that takes two parameters p1 and p2. Inside, calculate a simple similarity score by dividing the number of common words by the total unique words in both prompts. Return True if the similarity is greater than or equal to similarity_threshold, otherwise False.

LangChain

prompt_v1 = "What is the weather like today?"
prompt_v2 = "Can you tell me the weather forecast for today?"
similarity_threshold = 0.8
# Your code here

Hint

Use sets to find common and unique words. Compare similarity to similarity_threshold.

4

COMPLETION: Call the comparison function

Add a line that calls compare_prompts with prompt_v1 and prompt_v2 as arguments and assigns the result to a variable called are_similar.

LangChain

prompt_v1 = "What is the weather like today?"
prompt_v2 = "Can you tell me the weather forecast for today?"
similarity_threshold = 0.8

def compare_prompts(p1, p2):
    words_p1 = set(p1.lower().split())
    words_p2 = set(p2.lower().split())
    common_words = words_p1.intersection(words_p2)
    total_unique_words = words_p1.union(words_p2)
    similarity = len(common_words) / len(total_unique_words)
    return similarity >= similarity_threshold

# Your code here

Hint

Assign the function call result to are_similar.

Practice

1. What is the main purpose of comparing different prompt versions in Langchain?

easy

A. To find the best wording that improves AI task results

B. To increase the number of API calls

C. To reduce the size of the prompt template

D. To change the programming language used

Solution

1. Step 1: Understand the goal of prompt comparison

   Comparing prompt versions helps identify which wording or structure yields better AI responses.

2. Step 2: Eliminate unrelated options

   Increasing API calls, reducing prompt size, or changing language do not relate to improving prompt effectiveness.

3. Final Answer:

   To find the best wording that improves AI task results -> Option A

4. Quick Check:

   Comparing prompts = find best wording [OK]

Hint: Focus on improving AI output quality, not technical details [OK]

Common Mistakes:

• Thinking prompt comparison reduces API calls
• Confusing prompt size with prompt quality
• Assuming language change is the goal

2. Which of the following is the correct way to create a PromptTemplate in Langchain?

easy

A. PromptTemplate(prompt="Hello {name}", args=["name"])

B. PromptTemplate(name="Hello {name}", variables=["name"])

C. PromptTemplate(text="Hello {name}", inputs=["name"])

D. PromptTemplate(template="Hello {name}", input_variables=["name"])

Solution

1. Step 1: Recall PromptTemplate syntax

   The correct constructor uses 'template' for the prompt text and 'input_variables' for placeholders.

2. Step 2: Check each option

   Only PromptTemplate(template="Hello {name}", input_variables=["name"]) uses 'template' and 'input_variables' correctly; others use wrong parameter names.

3. Final Answer:

   PromptTemplate(template="Hello {name}", input_variables=["name"]) -> Option D

4. Quick Check:

   Correct parameters = template + input_variables [OK]

Hint: Remember: 'template' and 'input_variables' are required keys [OK]

Common Mistakes:

• Using 'name' instead of 'template' for prompt text
• Using 'variables' instead of 'input_variables'
• Confusing parameter names

3. Given the code below, what will be printed?

from langchain import PromptTemplate

prompt_v1 = PromptTemplate(template="Hello, {name}!", input_variables=["name"])
prompt_v2 = PromptTemplate(template="Hi {name}, how are you?", input_variables=["name"])

print(prompt_v1.format(name="Alice"))
print(prompt_v2.format(name="Alice"))

medium

A. Hello Alice!
Hi Alice, how are you?

B. Hello, Alice!
Hi Alice, how are you?

C. Hello, {name}!
Hi {name}, how are you?

D. Error: Missing input variable

Solution

1. Step 1: Understand PromptTemplate.format()

   The format method replaces placeholders with provided values, here 'name' is 'Alice'.

2. Step 2: Apply formatting to each prompt

   prompt_v1 becomes "Hello, Alice!" and prompt_v2 becomes "Hi Alice, how are you?".

3. Final Answer:

   Hello, Alice! Hi Alice, how are you? -> Option B

4. Quick Check:

   Formatted prompts show replaced names [OK]

Hint: Format replaces {name} with 'Alice' exactly [OK]

Common Mistakes:

• Ignoring commas or punctuation in output
• Printing raw template without formatting
• Assuming error without missing inputs

4. What is the error in the following code snippet?

from langchain import PromptTemplate

prompt = PromptTemplate(template="Hello, {user}!")
print(prompt.format(name="Bob"))

medium

A. Missing input_variables parameter in PromptTemplate

B. PromptTemplate cannot be imported from langchain

C. Using 'name' instead of 'user' in format call

D. No error, code runs fine

Solution

1. Step 1: Check PromptTemplate parameters

   While 'input_variables' is recommended, it is optional if placeholders are in template.

2. Step 2: Check format call variables

   The template expects 'user' but format is called with 'name', causing a KeyError.

3. Final Answer:

   Using 'name' instead of 'user' in format call -> Option C

4. Quick Check:

   Format keys must match template placeholders [OK]

Hint: Match format keys exactly to template placeholders [OK]

Common Mistakes:

• Assuming missing input_variables causes error
• Thinking import is wrong
• Ignoring variable name mismatch

5. You want to compare two prompt versions to see which generates a more polite greeting. You have these prompts:

prompt_v1 = PromptTemplate(template="Hey {name}, what's up?", input_variables=["name"])
prompt_v2 = PromptTemplate(template="Good day, {name}. How do you do?", input_variables=["name"])

Which approach best helps you compare their effectiveness?

hard

A. Format both prompts with the same name and print outputs side-by-side for review

B. Use only prompt_v1 since it is shorter and simpler

C. Change the input variable names to different ones for each prompt

D. Run prompt_v2 without formatting to see the raw template

Solution

1. Step 1: Understand comparison goal

   You want to see which prompt wording sounds more polite for the same input.

2. Step 2: Use consistent input and print both outputs

   Formatting both prompts with the same name and printing outputs side-by-side lets you compare wording clearly.

3. Final Answer:

   Format both prompts with the same name and print outputs side-by-side for review -> Option A

4. Quick Check:

   Compare outputs side-by-side for best prompt [OK]

Hint: Print both formatted prompts together to compare easily [OK]

Common Mistakes:

• Choosing only one prompt without comparison
• Changing input variable names inconsistently
• Not formatting prompts before comparing