Error handling and rate limits in Prompt Engineering / GenAI - Full Explanation

Introduction

Imagine using a service that suddenly stops working or slows down because too many people are using it at once. This problem happens often with online tools and services. To keep things running smoothly, systems use error handling and rate limits to manage problems and control how much users can access the service.

Explanation

Error Handling

Error handling is the way a system responds when something goes wrong, like a broken connection or invalid input. Instead of crashing or freezing, the system catches the problem and gives a clear message or tries to fix it. This helps users understand what happened and what to do next.

Error handling helps systems manage problems gracefully and keep users informed.

Types of Errors

Errors can be temporary, like a network glitch, or permanent, like a wrong password. Systems often classify errors to decide how to respond. For example, a temporary error might trigger a retry, while a permanent error shows a message to the user.

Knowing the type of error helps decide the best way to respond.

Rate Limits

Rate limits control how many times a user or program can access a service in a certain time. This prevents overload and keeps the service fair for everyone. If the limit is reached, the system blocks extra requests temporarily and tells the user to wait.

Rate limits protect services from being overwhelmed by too many requests.

Handling Rate Limit Errors

When a user hits a rate limit, the system usually sends a specific error message. Good systems tell users how long to wait before trying again. Some programs automatically pause and retry after this wait time to avoid errors.

Clear messages and wait times help users and programs handle rate limits smoothly.

Real World Analogy

Imagine a popular coffee shop that can only serve a certain number of customers at once. If too many people arrive, the shop asks some to wait outside until there is space. If the coffee machine breaks, the barista tells customers about the problem and suggests coming back later.

Error Handling → Barista explaining the coffee machine is broken and suggesting to come back later

Types of Errors → Temporary problem like the machine needing a quick fix versus permanent problem like no coffee beans

Rate Limits → Only allowing a certain number of customers inside the shop at one time

Handling Rate Limit Errors → Asking customers to wait outside and telling them how long before they can enter

Diagram

Diagram

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   User sends  │──────▶│  Service checks│──────▶│  Accept or    │
│   request     │       │  rate limits   │       │  reject       │
└───────────────┘       └───────────────┘       └───────────────┘
                                   │                      │
                                   │                      ▼
                                   │             ┌───────────────┐
                                   │             │  Error or      │
                                   │             │  Success      │
                                   │             └───────────────┘
                                   ▼                      │
                          ┌─────────────────┐            │
                          │  If rate limit   │◀───────────┘
                          │  exceeded, send  │
                          │  error message   │
                          └─────────────────┘

This diagram shows how a service checks user requests against rate limits and either accepts them or sends an error.

Key Facts

Error Handling → The process of managing problems in a system to avoid crashes and inform users.

Rate Limits → Rules that limit how many requests a user can make to a service in a set time.

Temporary Error → An error caused by a short-term issue that might resolve on retry.

Permanent Error → An error caused by a lasting problem that requires user action to fix.

Rate Limit Error → An error sent when a user exceeds the allowed number of requests.

Common Confusions

Believing all errors mean the system is broken.

Believing all errors mean the system is broken. Many errors are temporary or user-related and do not mean the entire system has failed.

Thinking rate limits block users permanently.

Thinking rate limits block users permanently. Rate limits only block users temporarily to prevent overload, and access is restored after waiting.

Assuming error messages always explain what to do next.

Assuming error messages always explain what to do next. Good error messages guide users, but some systems may give vague messages that need improvement.

Summary

Error handling helps systems respond to problems without crashing and keeps users informed.

Rate limits protect services by controlling how many requests users can make in a short time.

Clear error messages and wait times help users and programs handle rate limits smoothly.

Practice

1. What is the main purpose of using error handling in AI applications?

easy

A. To keep the app running smoothly even when problems happen

B. To speed up the AI model training process

C. To increase the number of requests sent to the server

D. To reduce the size of the AI model

Solution

1. Step 1: Understand error handling purpose

   Error handling is used to manage unexpected problems during app execution.

2. Step 2: Connect to AI app context

   In AI apps, error handling helps keep the app running smoothly despite issues.

3. Final Answer:

   To keep the app running smoothly even when problems happen -> Option A

4. Quick Check:

   Error handling = keep app running smoothly [OK]

Hint: Error handling means catching problems to avoid crashes [OK]

Common Mistakes:

• Thinking error handling speeds up training
• Confusing error handling with increasing requests
• Believing error handling reduces model size

2. Which Python syntax correctly catches an error when calling an AI API?

easy

A. try:
response = call_api()
except:
print('Error occurred')

B. catch:
response = call_api()
try:
print('Error occurred')

C. if error:
response = call_api()
else:
print('Error occurred')

D. error handling:
response = call_api()
except:
print('Error occurred')

Solution

1. Step 1: Identify correct try-except syntax

   Python uses try: block followed by except: to catch errors.

2. Step 2: Match syntax with options

   try: response = call_api() except: print('Error occurred') uses correct try-except structure; others use invalid keywords.

3. Final Answer:

   try:\n response = call_api()\nexcept:\n print('Error occurred') -> Option A

4. Quick Check:

   try-except syntax = try: response = call_api() except: print('Error occurred') [OK]

Hint: Remember Python uses try: and except: blocks [OK]

Common Mistakes:

• Using catch instead of except
• Using if error instead of try-except
• Writing invalid keywords like error handling:

3. What will the following Python code print if the API returns a rate limit error?

import time

try:
    response = call_api()
except RateLimitError:
    print('Rate limit hit, waiting...')
    time.sleep(10)
    response = call_api()
print('Done')

medium

A. Error: RateLimitError not caught

B. Done

C. Rate limit hit, waiting...

D. Rate limit hit, waiting...\nDone

Solution

1. Step 1: Understand try-except with RateLimitError

   If call_api() raises RateLimitError, except block runs printing message and waits 10 seconds.

2. Step 2: After waiting, call_api() runs again and then prints 'Done'

   So output includes the message and 'Done' on separate lines.

3. Final Answer:

   Rate limit hit, waiting...\nDone -> Option D

4. Quick Check:

   RateLimitError caught, message + Done printed [OK]

Hint: Exception caught prints message then continues [OK]

Common Mistakes:

• Assuming no message prints
• Thinking program crashes on rate limit
• Ignoring the second call_api() after sleep

4. Identify the error in this code snippet handling rate limits:

try:
    response = call_api()
except RateLimitError
    print('Too many requests')
    time.sleep(5)
    response = call_api()

medium

A. call_api() should not be retried

B. time.sleep() cannot be used in except block

C. Missing colon after except RateLimitError

D. print statement syntax is incorrect

Solution

1. Step 1: Check except syntax

   Python requires a colon ':' after except RateLimitError to start the block.

2. Step 2: Verify other parts

   time.sleep() is valid, retrying call_api() is allowed, print syntax is correct.

3. Final Answer:

   Missing colon after except RateLimitError -> Option C

4. Quick Check:

   except needs colon ':' [OK]

Hint: except lines always end with a colon ':' [OK]

Common Mistakes:

• Forgetting colon after except
• Thinking sleep() is invalid in except
• Believing retry is not allowed

5. You want to build an AI app that calls an API but respects rate limits by retrying after waiting. Which code snippet correctly implements this with error handling and exponential backoff?

hard

A. import time
wait = 1
for _ in range(3):
try:
response = call_api()
break
except RateLimitError:
time.sleep(wait)
wait *= 2

B. import time
wait = 1
while True:
try:
response = call_api()
break
except RateLimitError:
time.sleep(wait)
wait *= 2

C. import time
wait = 1
for _ in range(3):
try:
response = call_api()
except RateLimitError:
wait *= 2
time.sleep(wait)
else:
break

D. import time
wait = 1
while True:
try:
response = call_api()
except RateLimitError:
time.sleep(wait)
wait += 1
else:
break

Solution

1. Step 1: Understand exponential backoff with retries

   We want to retry after waiting, doubling wait time each failure, and stop on success.

2. Step 2: Analyze options for correct loop and break

   import time wait = 1 while True: try: response = call_api() break except RateLimitError: time.sleep(wait) wait *= 2 uses while True loop, tries call_api(), breaks on success, and doubles wait after RateLimitError.

3. Step 3: Check other options

   import time wait = 1 for _ in range(3): try: response = call_api() break except RateLimitError: time.sleep(wait) wait *= 2 breaks on success but uses for loop with fixed tries (less flexible). import time wait = 1 while True: try: response = call_api() except RateLimitError: time.sleep(wait) wait += 1 else: break increments wait linearly, not exponential. import time wait = 1 for _ in range(3): try: response = call_api() except RateLimitError: wait *= 2 time.sleep(wait) else: break doubles wait before sleep, but order is less clear.

4. Final Answer:

   import time wait = 1 while True: try: response = call_api() break except RateLimitError: time.sleep(wait) wait *= 2 -> Option B

5. Quick Check:

   Retry loop with exponential backoff = import time wait = 1 while True: try: response = call_api() break except RateLimitError: time.sleep(wait) wait *= 2 [OK]

Hint: Use while True with break and double wait after error [OK]

Common Mistakes:

• Using for loop limits retries too strictly
• Incrementing wait linearly instead of doubling
• Not breaking loop on success