Token usage and cost tracking in Agentic Ai - ML Experiment: Train & Evaluate

Experiment - Token usage and cost tracking

Problem:You are using an AI language model that charges based on the number of tokens processed. Currently, you do not track how many tokens your requests use or the cost incurred. This makes it hard to manage your budget and optimize usage.

Current Metrics:No token usage or cost tracking implemented; total cost unknown.

Issue:Without tracking tokens and cost, you risk overspending and cannot optimize your queries for cost efficiency.

Your Task

Implement a system to count tokens used per request and calculate the total cost based on a fixed price per 1000 tokens. The system should output token counts and cumulative cost after each request.

You must use the provided token counting method or a simple approximation.

Assume a fixed cost rate of $0.002 per 1000 tokens.

Do not change the AI model or request content.

Hint 1

Hint 2

Hint 3

Solution

Agentic_ai

import re

class TokenCostTracker:
    def __init__(self, cost_per_1000_tokens=0.002):
        self.total_tokens = 0
        self.total_cost = 0.0
        self.cost_per_1000_tokens = cost_per_1000_tokens

    def count_tokens(self, text):
        # Simple approximation: count words as tokens
        tokens = len(re.findall(r"\w+", text))
        return tokens

    def update(self, prompt, response):
        prompt_tokens = self.count_tokens(prompt)
        response_tokens = self.count_tokens(response)
        used_tokens = prompt_tokens + response_tokens
        self.total_tokens += used_tokens
        cost = (used_tokens / 1000) * self.cost_per_1000_tokens
        self.total_cost += cost
        return {
            "prompt_tokens": prompt_tokens,
            "response_tokens": response_tokens,
            "used_tokens": used_tokens,
            "total_tokens": self.total_tokens,
            "cost": round(cost, 6),
            "total_cost": round(self.total_cost, 6)
        }

# Example usage
tracker = TokenCostTracker()
prompt = "Hello, how are you today?"
response = "I am fine, thank you! How can I help you?"
usage1 = tracker.update(prompt, response)

prompt2 = "Tell me a joke."
response2 = "Why did the chicken cross the road? To get to the other side!"
usage2 = tracker.update(prompt2, response2)

print(usage1)
print(usage2)

Created a TokenCostTracker class to count tokens and track cost.

Implemented a simple token counting method based on word count.

Added methods to update total tokens and cost after each prompt-response pair.

Displayed token usage and cost details after each interaction.

Results Interpretation

Before: No token or cost tracking, total cost unknown.

After: Token usage and cost tracked per request. Example: First request used 15 tokens costing $0.000030, second request used 17 tokens costing $0.000034, total cost $0.000064.

Tracking token usage and cost helps manage AI usage budgets and optimize queries to reduce expenses.

Bonus Experiment

Extend the tracker to visualize token usage and cost over multiple requests using a simple line chart.

💡 Hint

Use matplotlib or any plotting library to plot total tokens and total cost after each request.