Rate limiting in Microservices - Architecture Diagram

System Overview - Rate limiting

This system controls how many requests a user or client can make to an API within a certain time. It protects the service from overload and abuse by limiting request rates. The system must be scalable and handle many users simultaneously.

Architecture Diagram

User
  |
  v
Load Balancer
  |
  v
API Gateway -- Rate Limiter -- Redis Cache
  |                      |
  v                      v
Microservices ---------> Database

Components

User

client

Sends requests to the system

Load Balancer

load_balancer

Distributes incoming requests evenly to API Gateway instances

API Gateway

api_gateway

Entry point for requests; enforces rate limiting before forwarding

Rate Limiter

service

Checks and enforces request limits per user using counters

Redis Cache

cache

Stores counters for requests per user with fast access

Microservices

service

Handles business logic after passing rate limiting

Database

database

Stores persistent data for the microservices

Request Flow - 10 Hops

User → Load Balancer

Load Balancer → API Gateway

API Gateway → Rate Limiter

Rate Limiter → Redis Cache

Redis Cache → Rate Limiter

Rate Limiter → API Gateway

API Gateway → Microservices

Microservices → Database

Microservices → API Gateway

API Gateway → User

Failure Scenario

Component Fails:Redis Cache

Impact:Rate limiter cannot check or update counters, so rate limiting may fail or become inaccurate, risking overload or blocking all requests.

Mitigation:Fallback to in-memory counters with limited accuracy or temporarily disable rate limiting with alerts; restore Redis quickly with replication and monitoring.

Architecture Quiz - 3 Questions

Which component is responsible for evenly distributing incoming user requests?

A Load Balancer

B API Gateway

C Rate Limiter

D Redis Cache

Design Principle

This architecture uses a centralized rate limiter with a fast cache to efficiently track request counts per user. It protects backend services from overload by rejecting excessive requests early. The load balancer and API gateway ensure scalability and control entry points. Using Redis for counters provides low latency and atomic increments, essential for accurate rate limiting.