Test cases for tool-using agents in Agentic AI

Introduction

Test cases help check if tool-using agents work correctly. They make sure the agent uses tools as expected.

When you want to verify an agent can call external tools properly.

When you need to check if the agent handles tool errors gracefully.

When you want to confirm the agent follows the right steps using tools.

When updating or improving the agent's tool usage abilities.

When debugging why an agent's tool use is not giving correct results.

Syntax

Agentic AI

def test_agent_tool_usage(agent, tool_input, expected_output):
    result = agent.use_tool(tool_input)
    assert result == expected_output, f"Expected {expected_output}, got {result}"

This is a simple test function to check tool usage.

Use assertions to compare expected and actual results.

Examples

Test if the agent returns the expected weather info when using a search tool.

Agentic AI

def test_agent_can_search(agent):
    query = "weather today"
    expected = "Sunny"
    result = agent.use_tool(query)
    assert result == expected

Test if the agent raises an error when given bad input to a tool.

Agentic AI

def test_agent_handles_tool_error(agent):
    bad_input = ""
    try:
        agent.use_tool(bad_input)
    except ValueError:
        pass
    else:
        assert False, "Agent did not raise error on bad input"

Sample Model

This program defines a simple agent that uses a tool to get weather info. It tests correct output and error handling.

Agentic AI

class SimpleAgent:
    def use_tool(self, input_text):
        if not input_text:
            raise ValueError("Input cannot be empty")
        if input_text == "weather today":
            return "Sunny"
        return "Unknown"

def test_agent_tool_usage():
    agent = SimpleAgent()
    # Test correct tool usage
    assert agent.use_tool("weather today") == "Sunny"
    # Test error handling
    try:
        agent.use_tool("")
    except ValueError as e:
        print(f"Caught error: {e}")
    else:
        print("Error not caught")

if __name__ == "__main__":
    test_agent_tool_usage()

Important Notes

Always test both normal and error cases for tool-using agents.

Use clear, simple inputs and expected outputs in tests.

Run tests often to catch bugs early.

Summary

Test cases check if agents use tools correctly and handle errors.

Write tests with expected inputs and outputs.

Testing helps keep agents reliable and easy to improve.