Test cases for tool-using agents in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of tool-using agents in AI?

Tool-using agents in AI are designed to:

A Learn from data without any external help

B Replace human decision-making completely without any tools

C Use external tools or APIs to enhance their problem-solving abilities

D Operate only within a fixed environment without interaction

💡 Hint

Think about how agents can improve their capabilities beyond their own programming.

Tool-using agents leverage external tools or APIs to perform tasks they cannot do alone, enhancing their problem-solving skills.

❓ Predict Output

intermediate

What output does this tool-using agent produce?

Consider this simplified Python code where an agent uses a calculator tool to add numbers:

Agentic AI

class CalculatorTool:
    def add(self, x, y):
        return x + y

class Agent:
    def __init__(self, tool):
        self.tool = tool
    def compute_sum(self, a, b):
        return self.tool.add(a, b)

calc = CalculatorTool()
agent = Agent(calc)
result = agent.compute_sum(5, 7)
print(result)

A None

B 57

C TypeError

D 12

💡 Hint

Check what the add method returns and how compute_sum uses it.

The add method returns the sum of 5 and 7, which is 12, so the agent prints 12.

❓ Model Choice

advanced

Which model architecture best suits a tool-using agent that must handle text and images?

You want to build a tool-using agent that can understand text commands and analyze images to decide which tool to use. Which model architecture is best?

A A multimodal transformer model that processes both text and images

B A simple feedforward neural network with only text input

C A convolutional neural network (CNN) for images only

D A recurrent neural network (RNN) for sequential text data only

💡 Hint

Think about models that can handle multiple types of data at once.

Multimodal transformers can process both text and images, making them ideal for agents that use multiple input types to decide tool usage.

❓ Metrics

advanced

Which metric best evaluates a tool-using agent's success in completing tasks?

You have an agent that uses external tools to complete tasks. Which metric best measures how well it completes these tasks?

A Task completion rate (percentage of tasks successfully finished)

B Loss value during training

C Accuracy of the agent's internal neural network weights

D Number of tools available to the agent

💡 Hint

Focus on the agent's real-world performance, not internal training stats.

Task completion rate directly measures how many tasks the agent finishes successfully, reflecting its practical effectiveness.

🔧 Debug

expert

Why does this tool-using agent code raise an AttributeError?

Examine the code below where an agent tries to use a tool but fails:

Agentic AI

class Tool:
    def execute(self, command):
        return f"Executed {command}"

class Agent:
    def __init__(self):
        self.tool = None
    def use_tool(self, cmd):
        return self.tool.execute(cmd)

agent = Agent()
output = agent.use_tool('run')
print(output)

A Because <code>Agent</code> class is missing an <code>execute</code> method

B Because <code>self.tool</code> is None and has no <code>execute</code> method

C Because <code>use_tool</code> method is missing a return statement

D Because <code>execute</code> method is misspelled

💡 Hint

Check what self.tool is when use_tool is called.

The agent's tool attribute is None, so calling execute on None raises an AttributeError.

Practice

1. What is the main purpose of writing test cases for tool-using agents?

easy

A. To add more tools to the agent

B. To make agents run faster

C. To check if agents use tools correctly and handle errors

D. To reduce the size of the agent's code

Solution

1. Step 1: Understand the role of test cases

   Test cases are designed to verify that the agent behaves as expected, especially when using tools.

2. Step 2: Identify the main goal for tool-using agents

   For agents that use tools, tests ensure they use these tools correctly and handle any errors gracefully.

3. Final Answer:

   To check if agents use tools correctly and handle errors -> Option C

4. Quick Check:

   Test cases purpose = check tool use and errors [OK]

Hint: Test cases verify correct tool use and error handling [OK]

Common Mistakes:

• Thinking test cases speed up agents
• Believing test cases reduce code size
• Assuming test cases add tools

2. Which of the following is the correct way to write a test case for a tool-using agent in Python?

easy

A. test agent tool: assert agent.use_tool('calculator', '2+2') == 4

B. def test_agent_tool(): assert agent.use_tool('calculator', '2+2') == 4

C. def test_agent_tool: assert agent.use_tool('calculator', '2+2') == 4

D. def test_agent_tool() assert agent.use_tool('calculator', '2+2') == 4

Solution

1. Step 1: Check Python function syntax

   Python test functions start with 'def', have parentheses, and a colon at the end.

2. Step 2: Verify assertion syntax

   The assert statement must be inside the function and correctly compare expected output.

3. Final Answer:

   def test_agent_tool(): assert agent.use_tool('calculator', '2+2') == 4 -> Option B

4. Quick Check:

   Correct Python test function syntax = def test_agent_tool(): assert agent.use_tool('calculator', '2+2') == 4 [OK]

Hint: Remember Python functions need parentheses and colon [OK]

Common Mistakes:

• Omitting parentheses in function definition
• Missing colon after function header
• Incorrect assert statement placement

3. Given this test case code snippet, what will be the output if the agent returns 5 instead of 4?

def test_agent_tool():
    result = agent.use_tool('calculator', '2+2')
    assert result == 4
    print('Test passed')

medium

A. Test passed

B. SyntaxError

C. No output

D. AssertionError

Solution

1. Step 1: Understand assert behavior

   If the assert condition is false, Python raises an AssertionError and stops execution.

2. Step 2: Check the test condition

   The test expects result == 4, but agent returns 5, so assert fails.

3. Final Answer:

   AssertionError -> Option D

4. Quick Check:

   Assert fails if values differ = AssertionError [OK]

Hint: Assert fails if expected and actual differ [OK]

Common Mistakes:

• Thinking print runs after failed assert
• Confusing AssertionError with SyntaxError
• Assuming no output on failure

4. Identify the error in this test case for a tool-using agent:

def test_agent_tool():
    result = agent.use_tool('search', 'weather today')
    assert result = 'sunny'
    print('Test passed')

medium

A. Using '=' instead of '==' in assert

B. Missing parentheses in print

C. Wrong function name

D. Agent tool name is invalid

Solution

1. Step 1: Check assert statement syntax

   In Python, '=' is for assignment, '==' is for comparison. Assert needs '==' to compare values.

2. Step 2: Verify other parts

   Print has parentheses, function name is valid, and tool name is plausible.

3. Final Answer:

   Using '=' instead of '==' in assert -> Option A

4. Quick Check:

   Assert needs '==' for comparison [OK]

Hint: Assert compares with '==', not '=' [OK]

Common Mistakes:

• Confusing assignment '=' with comparison '=='
• Ignoring syntax errors in assert
• Assuming print needs no parentheses

5. You want to test an agent that uses a calculator tool to handle multiple expressions. Which test case best checks if the agent correctly handles both valid and invalid inputs?

hard

A. def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', 'abc') == 'error'

B. def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', '3/0') == 0

C. def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', '') == ''

D. def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', null) == null"

Solution

1. Step 1: Check valid input test

   All options test '3*3' == 9 correctly, which is good for valid input.

2. Step 2: Check invalid input handling

   def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', 'abc') == 'error' expects 'abc' input to return 'error', which correctly tests error handling. Others expect incorrect or unclear outputs.

3. Final Answer:

   def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', 'abc') == 'error' -> Option A

4. Quick Check:

   Test valid and invalid inputs properly = def test_calc(): assert agent.use_tool('calculator', '3*3') == 9; assert agent.use_tool('calculator', 'abc') == 'error' [OK]

Hint: Test both valid and invalid inputs explicitly [OK]

Common Mistakes:

• Expecting wrong output for invalid input
• Not testing error cases
• Assuming empty or null inputs return themselves