Defining tool schemas and descriptions in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Defining tool schemas and descriptions

Problem:You want to create clear and correct tool schemas and descriptions for an AI agent to use tools effectively. Currently, the tool schemas are incomplete or unclear, causing the AI to misunderstand how to use the tools.

Current Metrics:Tool usage success rate: 60%, Tool error rate: 30%

Issue:The AI agent often fails to use tools correctly because the schemas lack clear input/output definitions and descriptions.

Your Task

Improve the tool schemas and descriptions so that the AI agent's tool usage success rate increases to at least 85% and error rate drops below 10%.

You cannot change the AI agent's core code or learning algorithm.

You can only modify the tool schemas and their descriptions.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Agentic AI

tools = [
    {
        "name": "search",
        "description": "Search the internet for information based on a query string.",
        "parameters": {
            "query": {
                "type": "string",
                "description": "The search query string to find relevant information. Example: 'weather today in New York'"
            }
        },
        "returns": {
            "type": "string",
            "description": "A summary of the search results relevant to the query."
        }
    },
    {
        "name": "calculator",
        "description": "Perform basic arithmetic calculations.",
        "parameters": {
            "expression": {
                "type": "string",
                "description": "A mathematical expression to evaluate. Example: '2 + 2 * 3'"
            }
        },
        "returns": {
            "type": "number",
            "description": "The result of the evaluated expression."
        }
    }
]

# Example usage:
# The AI agent reads the schema to know it must provide a 'query' string to 'search' and expects a string summary back.
# For 'calculator', it must provide an 'expression' string and expects a numeric result.

Added clear 'description' fields for each tool explaining its purpose.

Defined 'parameters' with explicit 'type' and 'description' including usage examples.

Specified 'returns' type and description for expected output.

Ensured all schemas are consistent and easy to understand.

Results Interpretation

Before: Success rate 60%, Error rate 30%
After: Success rate 88%, Error rate 8%

Clear and complete tool schemas with descriptive inputs and outputs help AI agents understand how to use tools correctly, reducing errors and improving performance.

Bonus Experiment

Try adding example inputs and outputs directly in the tool schemas to further improve clarity.

💡 Hint

Include sample JSON objects showing what input the AI should provide and what output it can expect.