Defining tool schemas and descriptions in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Defining tool schemas and descriptions

Problem:You want to create clear and correct tool schemas and descriptions for an AI agent to use tools effectively. Currently, the tool schemas are incomplete or unclear, causing the AI to misunderstand how to use the tools.

Current Metrics:Tool usage success rate: 60%, Tool error rate: 30%

Issue:The AI agent often fails to use tools correctly because the schemas lack clear input/output definitions and descriptions.

Your Task

Improve the tool schemas and descriptions so that the AI agent's tool usage success rate increases to at least 85% and error rate drops below 10%.

You cannot change the AI agent's core code or learning algorithm.

You can only modify the tool schemas and their descriptions.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Agentic AI

tools = [
    {
        "name": "search",
        "description": "Search the internet for information based on a query string.",
        "parameters": {
            "query": {
                "type": "string",
                "description": "The search query string to find relevant information. Example: 'weather today in New York'"
            }
        },
        "returns": {
            "type": "string",
            "description": "A summary of the search results relevant to the query."
        }
    },
    {
        "name": "calculator",
        "description": "Perform basic arithmetic calculations.",
        "parameters": {
            "expression": {
                "type": "string",
                "description": "A mathematical expression to evaluate. Example: '2 + 2 * 3'"
            }
        },
        "returns": {
            "type": "number",
            "description": "The result of the evaluated expression."
        }
    }
]

# Example usage:
# The AI agent reads the schema to know it must provide a 'query' string to 'search' and expects a string summary back.
# For 'calculator', it must provide an 'expression' string and expects a numeric result.

Added clear 'description' fields for each tool explaining its purpose.

Defined 'parameters' with explicit 'type' and 'description' including usage examples.

Specified 'returns' type and description for expected output.

Ensured all schemas are consistent and easy to understand.

Results Interpretation

Before: Success rate 60%, Error rate 30%
After: Success rate 88%, Error rate 8%

Clear and complete tool schemas with descriptive inputs and outputs help AI agents understand how to use tools correctly, reducing errors and improving performance.

Bonus Experiment

Try adding example inputs and outputs directly in the tool schemas to further improve clarity.

💡 Hint

Include sample JSON objects showing what input the AI should provide and what output it can expect.

Practice

1. What is the main purpose of defining a tool schema in agentic AI?

easy

A. To describe what the tool does and what inputs it needs

B. To write the tool's source code

C. To train the AI model

D. To store user data securely

Solution

1. Step 1: Understand the role of tool schemas

   Tool schemas explain what a tool does and what inputs it requires for proper use.

2. Step 2: Differentiate from other tasks

   Writing code, training models, or storing data are separate tasks not covered by schemas.

3. Final Answer:

   To describe what the tool does and what inputs it needs -> Option A

4. Quick Check:

   Tool schema = tool description + inputs [OK]

Hint: Schemas explain tool purpose and inputs clearly [OK]

Common Mistakes:

• Confusing schema with code implementation
• Thinking schema trains the AI
• Assuming schema stores data

2. Which of the following is the correct way to define a tool schema description in JSON format?

easy

A. {"name": search, "desc": "Finds info", inputs: {query: string}}

B. {"tool": search, description: Finds info, inputs: query string}

C. {"name": "search", "description": "Finds information", "inputs": {"query": "string"}}

D. {name: "search", description: "Finds information", inputs: {query: string}}

Solution

1. Step 1: Check JSON syntax correctness

   {"name": "search", "description": "Finds information", "inputs": {"query": "string"}} uses proper JSON with keys and string values quoted correctly.

2. Step 2: Verify key names and structure

   {"name": "search", "description": "Finds information", "inputs": {"query": "string"}} uses "name", "description", and "inputs" keys properly with correct value types.

3. Final Answer:

   {"name": "search", "description": "Finds information", "inputs": {"query": "string"}} -> Option C

4. Quick Check:

   Valid JSON with quoted keys and strings = {"name": "search", "description": "Finds information", "inputs": {"query": "string"}} [OK]

Hint: Look for quoted keys and string values in JSON [OK]

Common Mistakes:

• Missing quotes around keys or strings
• Using unquoted identifiers
• Incorrect key names or structure

3. Given this tool schema snippet:

{"name": "calculator", "description": "Performs math operations", "inputs": {"operation": "string", "x": "number", "y": "number"}}

What would be the expected output if the tool is called with inputs {"operation": "add", "x": 5, "y": 3}?

medium

A. 8

B. "add"

C. 53

D. Error: missing inputs

Solution

1. Step 1: Understand the tool schema inputs

   The tool expects an operation (like add) and two numbers x and y.

2. Step 2: Apply the operation to inputs

   Adding 5 and 3 results in 8 as output.

3. Final Answer:

   8 -> Option A

4. Quick Check:

   5 + 3 = 8 [OK]

Hint: Match inputs to operation and compute result [OK]

Common Mistakes:

• Returning operation string instead of result
• Concatenating numbers as strings
• Assuming missing inputs cause error

4. You have this tool schema:

{"name": "translator", "description": "Translates text", "inputs": {"text": "string", "target_lang": "string"}}

Which error is present in this schema?

medium

A. Description is too long

B. Name key is missing

C. Input types should be numbers

D. Missing closing brace for inputs object

Solution

1. Step 1: Check JSON syntax carefully

   The inputs object is opened but not closed with a } before the schema ends.

2. Step 2: Verify other keys

   Name key exists, description length is fine, input types as strings are correct.

3. Final Answer:

   Missing closing brace for inputs object -> Option D

4. Quick Check:

   Unclosed braces cause JSON errors [OK]

Hint: Count opening and closing braces carefully [OK]

Common Mistakes:

• Ignoring missing braces
• Thinking input types must be numbers
• Assuming description length matters

5. You want to define a tool schema for a weather forecast tool that takes a city name and date, and returns temperature and conditions. Which schema correctly describes this tool?

hard

A. {"name": "weather", "description": "Forecast tool", "inputs": {"city": "string"}, "outputs": {"temp": "string", "cond": "string"}}

B. {"name": "weather", "description": "Provides weather forecast", "inputs": {"city": "string", "date": "string"}, "outputs": {"temperature": "number", "conditions": "string"}}

C. {"name": "weather", "description": "Gives weather", "inputs": {"location": "string", "day": "date"}}

D. {"name": "weather", "inputs": {"city": "string", "date": "string"}}

Solution

1. Step 1: Check completeness of schema

   {"name": "weather", "description": "Provides weather forecast", "inputs": {"city": "string", "date": "string"}, "outputs": {"temperature": "number", "conditions": "string"}} includes name, description, inputs (city and date), and outputs (temperature and conditions) as required.

2. Step 2: Verify input and output types

   Inputs are strings (city, date), outputs are number and string, matching expected data.

3. Final Answer:

   {"name": "weather", "description": "Provides weather forecast", "inputs": {"city": "string", "date": "string"}, "outputs": {"temperature": "number", "conditions": "string"}} -> Option B

4. Quick Check:

   Complete schema with inputs and outputs = {"name": "weather", "description": "Provides weather forecast", "inputs": {"city": "string", "date": "string"}, "outputs": {"temperature": "number", "conditions": "string"}} [OK]

Hint: Include inputs and outputs with correct types [OK]

Common Mistakes:

• Omitting outputs section
• Using wrong input/output keys or types
• Leaving out description