Parallel tool execution in Agentic AI - ML Experiment: Train & Evaluate

Experiment - Parallel tool execution

Problem:You have an AI agent that uses multiple tools to answer questions. Currently, the agent runs these tools one after another, which slows down the response time.

Current Metrics:Average response time per query: 12 seconds; Accuracy: 90%

Issue:The agent's response time is too long because tools are executed sequentially, causing delays.

Your Task

Reduce the average response time to under 5 seconds by running tools in parallel, while maintaining accuracy above 88%.

You cannot reduce the number of tools used.

You must keep the same tool outputs and accuracy.

Use parallel execution methods supported by the agentic AI framework.

Hint 1

Hint 2

Hint 3

Solution

Agentic AI

import asyncio

class Tool:
    def __init__(self, name, delay, output):
        self.name = name
        self.delay = delay
        self.output = output

    async def run(self):
        await asyncio.sleep(self.delay)  # Simulate tool processing time
        return f"{self.name} output: {self.output}"

async def run_tools_in_parallel(tools):
    tasks = [tool.run() for tool in tools]
    results = await asyncio.gather(*tasks)
    return results

async def main():
    tools = [
        Tool("ToolA", 5, "dataA"),
        Tool("ToolB", 4, "dataB"),
        Tool("ToolC", 3, "dataC")
    ]
    results = await run_tools_in_parallel(tools)
    combined_result = " | ".join(results)
    print(f"Combined tool outputs: {combined_result}")

if __name__ == "__main__":
    import time
    start = time.time()
    asyncio.run(main())
    end = time.time()
    print(f"Total execution time: {end - start:.2f} seconds")

Replaced sequential tool execution with asynchronous parallel execution using asyncio.

Created async run method for each tool to simulate processing delay.

Used asyncio.gather to run all tools concurrently and collect outputs.

Combined outputs after all tools finished to maintain accuracy.

Results Interpretation

Before: Response time = 12 seconds, Accuracy = 90%

After: Response time = 5 seconds, Accuracy = 90%

Running multiple tools in parallel can significantly reduce response time without losing accuracy, demonstrating the power of asynchronous execution in AI agents.

Bonus Experiment

Try implementing parallel tool execution using multithreading instead of asyncio.

💡 Hint

Use Python's concurrent.futures.ThreadPoolExecutor to run tool calls concurrently and collect results.