Handling rate limits and errors in LangChain

Introduction

Handling rate limits and errors helps your program keep working smoothly even when the service is busy or something goes wrong.

When calling an API that limits how many requests you can make in a short time.

When you want to retry a request if it fails temporarily.

When you want to show a friendly message if something breaks.

When you want to avoid your program crashing because of unexpected errors.

Syntax

LangChain

from langchain.llms import OpenAI
import time

llm = OpenAI()
while True:
    try:
        response = llm("Hello!")
        break
    except Exception as error:
        if "rate limit" in str(error).lower():
            print("Rate limit hit, waiting before retrying...")
            time.sleep(5)
        else:
            raise

Use try-except blocks to catch errors and decide what to do.

Sleep and retry on rate limit errors.

Examples

This example waits 3 seconds and retries if a rate limit error happens.

LangChain

from langchain.llms import OpenAI
import time

llm = OpenAI()
while True:
    try:
        print(llm("Say hi"))
        break
    except Exception as error:
        if "rate limit" in str(error).lower():
            print("Waiting 3 seconds due to rate limit...")
            time.sleep(3)
        else:
            raise

This example just logs errors without retrying.

LangChain

from langchain.llms import OpenAI
from langchain.callbacks.base import BaseCallbackHandler

class LogErrorHandler(BaseCallbackHandler):
    def on_llm_error(self, error, **kwargs):
        print(f"Error caught: {error}")
        return False

llm = OpenAI(callbacks=[LogErrorHandler()])
print(llm("Hello"))

Sample Program

This program tries up to 3 times to get a joke from the OpenAI model. If it hits a rate limit, it waits 5 seconds and tries again. Other errors are printed and stop retries.

LangChain

from langchain.llms import OpenAI
import time

llm = OpenAI()
max_attempts = 3
for attempt in range(max_attempts):
    try:
        response = llm("Tell me a joke.")
        print("Response:", response)
        break
    except Exception as e:
        error_message = str(e).lower()
        if "rate limit" in error_message:
            print("Rate limit reached. Waiting 5 seconds before retrying...")
            time.sleep(5)
        else:
            print(f"Other error: {e}")
            break
else:
    print("Failed after retries")

Important Notes

Always handle rate limits to avoid your app stopping unexpectedly.

Use small wait times to be polite to the API and avoid long delays.

Logging errors helps you understand what went wrong.

Summary

Handling rate limits keeps your app running smoothly.

Use try-except with loops in Langchain to catch and retry errors.

Wait a bit before retrying to respect API limits.