Rate limiting and budget controls in Agentic Ai

Introduction

Rate limiting and budget controls help manage how much and how often an AI system can be used. This keeps the system fair and prevents overuse.

When you want to stop too many requests from one user in a short time.

When you need to keep your AI service costs under control.

When you want to make sure all users get a fair chance to use the AI.

When you want to avoid system overload or crashes.

When you want to track and limit usage based on a monthly or daily budget.

Syntax

Agentic_ai

rate_limit = RateLimiter(max_requests=100, time_window=60)  # 100 requests per 60 seconds
budget_control = BudgetController(max_cost=50.0)  # $50 max spending

if rate_limit.allow_request(user_id):
    if budget_control.can_spend(cost):
        response = ai_model.process(request)
        budget_control.record_spend(cost)
    else:
        response = 'Budget exceeded'
else:
    response = 'Rate limit exceeded'

RateLimiter controls how many requests can happen in a set time.

BudgetController tracks spending and stops requests if the budget is used up.

Examples

This example limits a user to 10 requests per minute and a $20 budget, spending $1.5 per request.

Agentic_ai

rate_limit = RateLimiter(max_requests=10, time_window=60)  # 10 requests per minute
budget_control = BudgetController(max_cost=20.0)  # $20 max

# Check before processing
if rate_limit.allow_request('user123') and budget_control.can_spend(1.5):
    # Process AI request
    budget_control.record_spend(1.5)

This example shows how requests beyond the limit are blocked within the time window.

Agentic_ai

rate_limit = RateLimiter(max_requests=5, time_window=10)  # 5 requests per 10 seconds

for i in range(7):
    if rate_limit.allow_request('user123'):
        print(f'Request {i+1} allowed')
    else:
        print(f'Request {i+1} blocked')

Sample Program

This program simulates 5 requests from one user. It allows up to 3 requests every 5 seconds and a total spending of $5. Each request costs $2. It prints if each request is allowed or blocked.

Agentic_ai

import time

class RateLimiter:
    def __init__(self, max_requests, time_window):
        self.max_requests = max_requests
        self.time_window = time_window
        self.requests = {}

    def allow_request(self, user_id):
        now = time.time()
        if user_id not in self.requests:
            self.requests[user_id] = []
        # Remove old requests
        self.requests[user_id] = [t for t in self.requests[user_id] if now - t < self.time_window]
        if len(self.requests[user_id]) < self.max_requests:
            self.requests[user_id].append(now)
            return True
        return False

class BudgetController:
    def __init__(self, max_cost):
        self.max_cost = max_cost
        self.spent = 0.0

    def can_spend(self, cost):
        return (self.spent + cost) <= self.max_cost

    def record_spend(self, cost):
        self.spent += cost

# Setup
rate_limit = RateLimiter(max_requests=3, time_window=5)  # 3 requests per 5 seconds
budget_control = BudgetController(max_cost=5.0)  # $5 max

user_id = 'user1'
request_cost = 2.0

for i in range(5):
    if rate_limit.allow_request(user_id):
        if budget_control.can_spend(request_cost):
            budget_control.record_spend(request_cost)
            print(f'Request {i+1}: Allowed, spent ${budget_control.spent}')
        else:
            print(f'Request {i+1}: Blocked - Budget exceeded')
    else:
        print(f'Request {i+1}: Blocked - Rate limit exceeded')
    time.sleep(1)

Important Notes

Rate limiting helps protect your AI system from too many requests at once.

Budget controls help you avoid unexpected costs by limiting spending.

Both controls can be combined for better management.

Summary

Rate limiting controls how often users can make requests.

Budget controls limit how much money can be spent on AI usage.

Using both keeps your AI system fair, stable, and cost-effective.