Rate limiting and budget controls in Agentic AI

Introduction

Rate limiting and budget controls help manage how much and how often an AI system can be used. This keeps the system fair and prevents overuse.

When you want to stop too many requests from one user in a short time.

When you need to keep your AI service costs under control.

When you want to make sure all users get a fair chance to use the AI.

When you want to avoid system overload or crashes.

When you want to track and limit usage based on a monthly or daily budget.

Syntax

Agentic AI

rate_limit = RateLimiter(max_requests=100, time_window=60)  # 100 requests per 60 seconds
budget_control = BudgetController(max_cost=50.0)  # $50 max spending

if rate_limit.allow_request(user_id):
    if budget_control.can_spend(cost):
        response = ai_model.process(request)
        budget_control.record_spend(cost)
    else:
        response = 'Budget exceeded'
else:
    response = 'Rate limit exceeded'

RateLimiter controls how many requests can happen in a set time.

BudgetController tracks spending and stops requests if the budget is used up.

Examples

This example limits a user to 10 requests per minute and a $20 budget, spending $1.5 per request.

Agentic AI

rate_limit = RateLimiter(max_requests=10, time_window=60)  # 10 requests per minute
budget_control = BudgetController(max_cost=20.0)  # $20 max

# Check before processing
if rate_limit.allow_request('user123') and budget_control.can_spend(1.5):
    # Process AI request
    budget_control.record_spend(1.5)

This example shows how requests beyond the limit are blocked within the time window.

Agentic AI

rate_limit = RateLimiter(max_requests=5, time_window=10)  # 5 requests per 10 seconds

for i in range(7):
    if rate_limit.allow_request('user123'):
        print(f'Request {i+1} allowed')
    else:
        print(f'Request {i+1} blocked')

Sample Model

This program simulates 5 requests from one user. It allows up to 3 requests every 5 seconds and a total spending of $5. Each request costs $2. It prints if each request is allowed or blocked.

Agentic AI

import time

class RateLimiter:
    def __init__(self, max_requests, time_window):
        self.max_requests = max_requests
        self.time_window = time_window
        self.requests = {}

    def allow_request(self, user_id):
        now = time.time()
        if user_id not in self.requests:
            self.requests[user_id] = []
        # Remove old requests
        self.requests[user_id] = [t for t in self.requests[user_id] if now - t < self.time_window]
        if len(self.requests[user_id]) < self.max_requests:
            self.requests[user_id].append(now)
            return True
        return False

class BudgetController:
    def __init__(self, max_cost):
        self.max_cost = max_cost
        self.spent = 0.0

    def can_spend(self, cost):
        return (self.spent + cost) <= self.max_cost

    def record_spend(self, cost):
        self.spent += cost

# Setup
rate_limit = RateLimiter(max_requests=3, time_window=5)  # 3 requests per 5 seconds
budget_control = BudgetController(max_cost=5.0)  # $5 max

user_id = 'user1'
request_cost = 2.0

for i in range(5):
    if rate_limit.allow_request(user_id):
        if budget_control.can_spend(request_cost):
            budget_control.record_spend(request_cost)
            print(f'Request {i+1}: Allowed, spent ${budget_control.spent}')
        else:
            print(f'Request {i+1}: Blocked - Budget exceeded')
    else:
        print(f'Request {i+1}: Blocked - Rate limit exceeded')
    time.sleep(1)

Important Notes

Rate limiting helps protect your AI system from too many requests at once.

Budget controls help you avoid unexpected costs by limiting spending.

Both controls can be combined for better management.

Summary

Rate limiting controls how often users can make requests.

Budget controls limit how much money can be spent on AI usage.

Using both keeps your AI system fair, stable, and cost-effective.

Practice

1. What is the main purpose of rate limiting in an AI system?

easy

A. To control how often users can make requests

B. To increase the speed of AI responses

C. To improve the accuracy of AI predictions

D. To store more user data for training

Solution

1. Step 1: Understand rate limiting concept

   Rate limiting is about controlling the number of requests a user can make in a time period.

2. Step 2: Identify the main purpose

   This control helps prevent overload and keeps the system fair for all users.

3. Final Answer:

   To control how often users can make requests -> Option A

4. Quick Check:

   Rate limiting = control request frequency [OK]

Hint: Rate limiting means limiting request frequency [OK]

Common Mistakes:

• Confusing rate limiting with improving AI accuracy
• Thinking rate limiting increases speed
• Mixing rate limiting with data storage

2. Which of the following is the correct way to set a budget control in an AI usage system?

easy

A. budget = max(1000)

B. limit_budget = 'max 1000 dollars'

C. setBudget(1000 dollars)

D. budget_limit = 1000 # sets max money allowed

Solution

1. Step 1: Identify correct syntax for budget control

   Setting a budget limit usually involves assigning a numeric value to a variable representing money allowed.

2. Step 2: Check each option

   budget_limit = 1000 # sets max money allowed uses a clear assignment with a comment, which is correct syntax. Others use invalid syntax or unclear expressions.

3. Final Answer:

   budget_limit = 1000 # sets max money allowed -> Option D

4. Quick Check:

   Assign numeric budget limit = correct [OK]

Hint: Budget control is a numeric variable assignment [OK]

Common Mistakes:

• Using strings instead of numbers for budget
• Calling undefined functions like setBudget
• Using incorrect syntax like max(1000)

3. Given this code snippet controlling requests per minute:

requests = [1,1,1,1,1,1]
limit = 5
allowed = sum(requests[:limit])
print(allowed)

What will be the printed output?

medium

A. 1

B. 6

C. 5

D. Error

Solution

1. Step 1: Understand the code slicing and summing

   The code sums the first 5 elements of the list requests, each element is 1.

2. Step 2: Calculate the sum of first 5 elements

   Sum = 1+1+1+1+1 = 5

3. Final Answer:

   5 -> Option C

4. Quick Check:

   Sum first 5 ones = 5 [OK]

Hint: Sum first 5 elements of ones list = 5 [OK]

Common Mistakes:

• Summing all 6 elements instead of 5
• Confusing slicing syntax
• Expecting an error due to slicing

4. Find the error in this rate limiting code snippet:

max_requests = 10
requests_made = 0
if requests_made > max_requests:
    print("Limit reached")
else:
    requests_made += 1
    print("Request allowed")

medium

A. No error, code is correct

B. The condition should be >= instead of >

C. The print statements are reversed

D. requests_made should start at 1

Solution

1. Step 1: Analyze the condition for rate limiting

   The code blocks requests if requests_made is greater than max_requests, but it should block when equal too.

2. Step 2: Correct the condition

   Change > to >= to include the max_requests limit properly.

3. Final Answer:

   The condition should be >= instead of > -> Option B

4. Quick Check:

   Use >= to block at limit [OK]

Hint: Use >= to block requests at limit [OK]

Common Mistakes:

• Using > misses blocking at exact limit
• Starting requests_made at 1 is unnecessary
• Swapping print messages confuses logic

5. You want to limit AI usage to 1000 requests per day and a budget of $500. Which approach correctly combines rate limiting and budget control?

hard

A. Set daily_limit = 1000 and budget_limit = 500; check both before allowing requests

B. Set daily_limit = 1000; budget_limit is not needed if rate limiting is set

C. Only set budget_limit = 500; rate limiting is handled automatically

D. Set budget_limit = 1000 and daily_limit = 500; swap values for safety

Solution

1. Step 1: Understand the need for both controls

   Rate limiting controls request count; budget controls money spent. Both must be checked.

2. Step 2: Evaluate options for combining controls

   Set daily_limit = 1000 and budget_limit = 500; check both before allowing requests correctly sets both limits and checks them before allowing requests. Others ignore one control or swap values incorrectly.

3. Final Answer:

   Set daily_limit = 1000 and budget_limit = 500; check both before allowing requests -> Option A

4. Quick Check:

   Use both limits together for control [OK]

Hint: Always check both request count and budget before allowing [OK]

Common Mistakes:

• Ignoring budget when rate limiting is set
• Assuming budget controls requests automatically
• Swapping limit values causing confusion