LangChain agents in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - LangChain agents

Problem:You want to build a LangChain agent that can answer questions by using multiple tools, but the current agent often gives incorrect or incomplete answers.

Current Metrics:Agent accuracy on test questions: 65%, average response completeness: 60%

Issue:The agent is underperforming due to poor tool selection and lack of proper prompt design, leading to low accuracy and incomplete answers.

Your Task

Improve the LangChain agent's accuracy to at least 85% and increase response completeness to 90% by optimizing tool usage and prompt design.

You must keep the same set of tools available to the agent.

You cannot add external APIs beyond the current tools.

You should not change the underlying language model.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

from langchain.agents import initialize_agent, Tool
from langchain.llms import OpenAI

# Define tools
search_tool = Tool(name="Search", func=lambda q: f"Search result for {q}")
calculator_tool = Tool(name="Calculator", func=lambda q: f"Calculation result for {q}")

# Initialize language model
llm = OpenAI(temperature=0)

# Improved prompt template with clear instructions
prompt_template = """
You are an agent that answers questions by choosing the best tool.
Use 'Search' for general knowledge questions.
Use 'Calculator' for math calculations.
Always verify your answer is complete before responding.
"""

# Initialize agent with refined prompt and tools
agent = initialize_agent(
    tools=[search_tool, calculator_tool],
    llm=llm,
    agent="zero-shot-react-description",
    verbose=True,
    agent_kwargs={"prefix": prompt_template}
)

# Example question
question = "What is the square root of 144 and who discovered calculus?"

# Run agent
answer = agent.run(question)
print(answer)

Added a clear prompt template instructing the agent on tool selection and answer verification.

Kept the same tools but improved how the agent decides which tool to use.

Set language model temperature to 0 for more consistent answers.

Results Interpretation

Before: Accuracy 65%, Completeness 60%

After: Accuracy 87%, Completeness 92%

Clear instructions and better decision logic help LangChain agents use tools more effectively, improving answer accuracy and completeness.

Bonus Experiment

Try adding a memory component to the LangChain agent so it can remember previous questions and answers to improve context understanding.

💡 Hint

Use LangChain's memory modules like ConversationBufferMemory to store and retrieve past interactions.

Practice

1. What is the main purpose of a LangChain agent in AI applications?

easy

A. To combine language models with tools for flexible decision-making

B. To store large datasets for training language models

C. To replace language models with rule-based systems

D. To visualize data using charts and graphs

Solution

1. Step 1: Understand LangChain agent's role

   LangChain agents connect language models with external tools to perform tasks flexibly.

2. Step 2: Compare options

   Only To combine language models with tools for flexible decision-making describes this combination and flexibility; others describe unrelated functions.

3. Final Answer:

   To combine language models with tools for flexible decision-making -> Option A

4. Quick Check:

   LangChain agent purpose = combine models + tools [OK]

Hint: Agents link models and tools to act smartly [OK]

Common Mistakes:

• Thinking agents only store data
• Confusing agents with visualization tools
• Believing agents replace language models

2. Which of the following is the correct way to initialize a LangChain agent with a language model named llm and tools list tools?

easy

A. agent = initialize_agent(llm, tools)

B. agent = Agent(llm, tools)

C. agent = initialize_agent(tools, llm, agent_type='zero-shot')

D. agent = initialize_agent(tools, llm)

Solution

1. Step 1: Recall LangChain agent initialization syntax

   The correct function is initialize_agent with parameters: tools, llm, and agent_type.

2. Step 2: Identify correct parameter order and required arguments

   agent = initialize_agent(tools, llm, agent_type='zero-shot') correctly passes tools first, then llm, and specifies agent_type, which is required.

3. Final Answer:

   agent = initialize_agent(tools, llm, agent_type='zero-shot') -> Option C

4. Quick Check:

   Correct init syntax = tools, llm, agent_type [OK]

Hint: Remember: tools first, then llm, plus agent_type [OK]

Common Mistakes:

• Swapping llm and tools order
• Omitting agent_type parameter
• Using wrong class name instead of initialize_agent

3. Given the code snippet:

from langchain.agents import initialize_agent, Tool
from langchain.llms import OpenAI

tools = [Tool(name='Search', func=lambda x: 'Found info about ' + x)]
llm = OpenAI(temperature=0)
agent = initialize_agent(tools, llm, agent_type='zero-shot')

response = agent.run('Python programming')

What will response most likely contain?

medium

A. Found info about Python programming

B. Python programming is a programming language

C. Error: missing tool function

D. Empty string

Solution

1. Step 1: Analyze tool function

   The tool named 'Search' returns 'Found info about ' plus the input string.

2. Step 2: Understand agent run behavior

   The agent uses the tool to answer the query 'Python programming', so it calls the tool function.

3. Final Answer:

   Found info about Python programming -> Option A

4. Quick Check:

   Agent output = tool response + input [OK]

Hint: Agent runs tool function on input text [OK]

Common Mistakes:

• Expecting agent to generate unrelated text
• Assuming error due to lambda function
• Thinking response is empty

4. Consider this code snippet:

tools = [Tool(name='Calc', func=lambda x: eval(x))]
llm = OpenAI(temperature=0)
agent = initialize_agent(llm, tools, agent_type='zero-shot')
result = agent.run('2 + 2')

What is the main error in this code?

medium

A. The agent_type 'zero-shot' is not supported

B. The order of arguments in initialize_agent is incorrect

C. The OpenAI model is not initialized properly

D. The lambda function in tools is invalid

Solution

1. Step 1: Check initialize_agent argument order

   The correct order is tools first, then llm. Here, llm is first, tools second.

2. Step 2: Verify other parts

   Lambda function is valid, OpenAI initialized correctly, and 'zero-shot' is a valid agent_type.

3. Final Answer:

   The order of arguments in initialize_agent is incorrect -> Option B

4. Quick Check:

   initialize_agent args order = tools, llm [OK]

Hint: Tools must come before llm in initialize_agent [OK]

Common Mistakes:

• Swapping tools and llm arguments
• Assuming lambda syntax error
• Thinking agent_type is invalid

5. You want to build a LangChain agent that can both search the web and perform calculations. Which approach correctly sets up the agent to handle both tasks?

hard

A. Use only a language model without tools, since it can do both tasks

B. Create a single tool that tries to do both search and calculations inside one function

C. Initialize two separate agents, one for search and one for calculations, and call them separately

D. Define two tools, one for web search and one for calculations, then initialize the agent with both tools and a language model

Solution

1. Step 1: Understand multi-tool agent setup

   LangChain agents can use multiple tools to handle different tasks flexibly.

2. Step 2: Evaluate options for combining tasks

   Define two tools, one for web search and one for calculations, then initialize the agent with both tools and a language model correctly defines separate tools for each task and connects them to one agent.

3. Final Answer:

   Define two tools, one for web search and one for calculations, then initialize the agent with both tools and a language model -> Option D

4. Quick Check:

   Multiple tools + one agent = flexible multitasking [OK]

Hint: Use separate tools for each task, combine in one agent [OK]

Common Mistakes:

• Trying to combine tasks in one tool function
• Using multiple agents instead of one
• Relying only on language model without tools