API gateway concept in HLD - System Design Guide

Problem Statement

When multiple backend services serve different parts of an application, clients must call each service separately. This leads to complex client logic, increased latency, and difficulty managing cross-cutting concerns like authentication and rate limiting.

Solution

An API gateway acts as a single entry point that routes client requests to the appropriate backend services. It centralizes common tasks such as authentication, request routing, rate limiting, and response aggregation, simplifying client interactions and improving performance.

Architecture

Clients

Clients

↓

API Gateway

API Gateway

↓

Service

A

This diagram shows clients sending requests to a single API gateway, which routes them to multiple backend services.

Trade-offs

✓ Pros

→

Simplifies client architecture by providing a single endpoint.

→

Centralizes cross-cutting concerns like authentication and rate limiting.

→

Enables response aggregation to reduce client-server round trips.

→

Allows independent evolution of backend services without impacting clients.

✗ Cons

→

Introduces a single point of failure if not highly available.

→

Adds an extra network hop, potentially increasing latency.

→

Can become a performance bottleneck under heavy load.

→

Increases operational complexity due to gateway management.

Use when your system has multiple backend services and clients need a unified interface, especially if you require centralized security, monitoring, or request aggregation at scale above 1000 requests per second.

Avoid if your system has a single backend service or very low traffic (under 100 requests per second), where the added complexity and latency outweigh benefits.

Real World Examples

Netflix

Netflix uses an API gateway to route client requests to various microservices, enabling centralized authentication and request routing for their streaming platform.

Amazon

Amazon employs API gateways to unify access to diverse backend services, simplifying client interactions and enforcing security policies.

Uber

Uber uses API gateways to aggregate data from multiple services like ride requests, payments, and user profiles, providing a seamless client experience.

Code Example

The before code shows the client making separate calls to user and order services, increasing complexity and latency. The after code shows a single call to the API gateway, which internally routes and aggregates data, simplifying the client.

HLD

### Before: Client calls multiple services directly
import requests

def get_user_profile(user_id):
    user = requests.get(f"http://user-service/users/{user_id}").json()
    orders = requests.get(f"http://order-service/orders?user={user_id}").json()
    return {"user": user, "orders": orders}

### After: Client calls API gateway once
import requests

def get_user_profile(user_id):
    response = requests.get(f"http://api-gateway/profile/{user_id}")
    return response.json()

Alternatives

Client-side aggregation

Clients call multiple services directly and aggregate responses themselves.

Use when: Choose when clients are powerful and network latency is low, and you want to avoid adding an extra gateway layer.

Backend for Frontend (BFF)

Separate backend services are created for each client type to tailor APIs specifically.

Use when: Choose when different client types (mobile, web) require customized APIs and logic.

Summary

An API gateway provides a single entry point to multiple backend services, simplifying client interactions.

It centralizes common tasks like authentication, routing, and rate limiting to improve maintainability.

However, it introduces an extra network hop and potential bottleneck, so use it when system complexity and scale justify it.