API Gateway pattern in Microservices - System Design Exercise

Design: API Gateway Pattern Design

Design focuses on the API Gateway component and its interaction with backend microservices. Client-side implementation and microservices internal design are out of scope.

Functional Requirements

FR1: Provide a single entry point for all client requests to multiple backend microservices

FR2: Handle request routing to appropriate microservices based on API endpoints

FR3: Perform authentication and authorization before forwarding requests

FR4: Aggregate responses from multiple microservices when needed

FR5: Support request throttling and rate limiting to protect backend services

FR6: Enable protocol translation (e.g., from HTTP to gRPC) if required

FR7: Provide centralized logging and metrics collection for all requests

Non-Functional Requirements

NFR1: Must handle 10,000 concurrent client requests

NFR2: API response latency p99 should be under 300ms

NFR3: Availability target of 99.9% uptime (less than 8.77 hours downtime per year)

NFR4: Scalable to add more microservices without changing client code

NFR5: Secure handling of sensitive data and tokens

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

❓ Question 7

Key Components

API Gateway server or service

Authentication and Authorization module

Routing and Load Balancer

Request Aggregator

Rate Limiter and Throttling module

Logging and Monitoring system

Cache layer (optional)

Design Patterns

Reverse Proxy pattern

Backend for Frontend (BFF) pattern

Circuit Breaker pattern for fault tolerance

Bulkhead pattern to isolate failures

Token-based Authentication pattern

Reference Architecture

Client
  |
  v
API Gateway
  |  |  |  \
  v  v  v   v
Microservice A  Microservice B  Microservice C

Components:
- API Gateway handles all client requests
- Auth module inside gateway verifies tokens
- Router directs requests to correct microservice
- Aggregator combines responses if needed
- Rate limiter protects backend
- Logger collects request data

Components

API Gateway

Nginx/Envoy/Custom Node.js service

Single entry point that routes requests to microservices

Authentication Module

JWT verification library or OAuth server integration

Verify client identity and permissions before forwarding

Router

API Gateway routing rules or service mesh

Direct requests to appropriate microservice endpoints

Request Aggregator

Custom logic in gateway or separate service

Combine multiple microservice responses into one

Rate Limiter

Redis-based token bucket or leaky bucket algorithm

Limit request rate to protect backend services

Logging and Monitoring

ELK stack, Prometheus, Grafana

Collect logs and metrics for observability

Cache Layer (optional)

Redis or in-memory cache

Cache frequent responses to reduce backend load

Request Flow

1. Client sends request to API Gateway endpoint

2. Rate limiter checks request frequency and blocks if limit exceeded

3. API Gateway authenticates request using token or credentials

4. If authentication fails, gateway returns error immediately

5. Gateway routes request to the correct microservice based on URL path

6. If aggregation is needed, gateway calls multiple microservices

7. Gateway combines responses and sends back to client

8. Logging module records request and response details for monitoring

Database Schema

Not applicable as API Gateway does not store persistent data. Authentication data is managed by separate identity service. Cache layer may store key-value pairs temporarily.

Scaling Discussion

Bottlenecks

API Gateway becomes a single point of failure under high load

Authentication module may slow down request processing

Request aggregation increases latency and resource use

Rate limiter state storage can become a bottleneck

Logging large volumes of requests can overwhelm storage

Solutions

Deploy multiple API Gateway instances behind a load balancer for high availability

Use stateless token verification and cache validation results

Optimize aggregation logic and consider asynchronous aggregation if possible

Use distributed in-memory stores like Redis clusters for rate limiting

Implement log sampling and use scalable log storage solutions

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why a single entry point simplifies client interaction

Discuss authentication and security importance at the gateway

Highlight how routing and aggregation improve client experience

Mention rate limiting to protect backend services

Address scalability and fault tolerance strategies

Show awareness of monitoring and observability needs