API gateway concept in HLD - System Design Guide

Problem Statement

When multiple backend services serve different parts of an application, clients must call each service separately. This leads to complex client logic, increased latency, and difficulty managing cross-cutting concerns like authentication and rate limiting.

Solution

An API gateway acts as a single entry point that routes client requests to the appropriate backend services. It centralizes common tasks such as authentication, request routing, rate limiting, and response aggregation, simplifying client interactions and improving performance.

Architecture

Clients

Clients

↓

API Gateway

API Gateway

↓

Service

A

This diagram shows clients sending requests to a single API gateway, which routes them to multiple backend services.

Trade-offs

✓ Pros

→

Simplifies client architecture by providing a single endpoint.

→

Centralizes cross-cutting concerns like authentication and rate limiting.

→

Enables response aggregation to reduce client-server round trips.

→

Allows independent evolution of backend services without impacting clients.

✗ Cons

→

Introduces a single point of failure if not highly available.

→

Adds an extra network hop, potentially increasing latency.

→

Can become a performance bottleneck under heavy load.

→

Increases operational complexity due to gateway management.

Use when your system has multiple backend services and clients need a unified interface, especially if you require centralized security, monitoring, or request aggregation at scale above 1000 requests per second.

Avoid if your system has a single backend service or very low traffic (under 100 requests per second), where the added complexity and latency outweigh benefits.

Real World Examples

Netflix

Netflix uses an API gateway to route client requests to various microservices, enabling centralized authentication and request routing for their streaming platform.

Amazon

Amazon employs API gateways to unify access to diverse backend services, simplifying client interactions and enforcing security policies.

Uber

Uber uses API gateways to aggregate data from multiple services like ride requests, payments, and user profiles, providing a seamless client experience.

Code Example

The before code shows the client making separate calls to user and order services, increasing complexity and latency. The after code shows a single call to the API gateway, which internally routes and aggregates data, simplifying the client.

HLD

### Before: Client calls multiple services directly
import requests

def get_user_profile(user_id):
    user = requests.get(f"http://user-service/users/{user_id}").json()
    orders = requests.get(f"http://order-service/orders?user={user_id}").json()
    return {"user": user, "orders": orders}

### After: Client calls API gateway once
import requests

def get_user_profile(user_id):
    response = requests.get(f"http://api-gateway/profile/{user_id}")
    return response.json()

Alternatives

Client-side aggregation

Clients call multiple services directly and aggregate responses themselves.

Use when: Choose when clients are powerful and network latency is low, and you want to avoid adding an extra gateway layer.

Backend for Frontend (BFF)

Separate backend services are created for each client type to tailor APIs specifically.

Use when: Choose when different client types (mobile, web) require customized APIs and logic.

Summary

An API gateway provides a single entry point to multiple backend services, simplifying client interactions.

It centralizes common tasks like authentication, routing, and rate limiting to improve maintainability.

However, it introduces an extra network hop and potential bottleneck, so use it when system complexity and scale justify it.

Practice

1. What is the primary role of an API gateway in a system architecture?

easy

A. It replaces the need for backend services.

B. It stores all the data for the application.

C. It directly manages the database connections.

D. It acts as a single entry point for client requests to backend services.

Solution

1. Step 1: Understand the role of API gateway

   An API gateway serves as the main entry point for clients to access multiple backend services, simplifying communication.

2. Step 2: Eliminate incorrect roles

   Storing data, managing database connections, or replacing backend services are not responsibilities of an API gateway.

3. Final Answer:

   It acts as a single entry point for client requests to backend services. -> Option D

4. Quick Check:

   API gateway = single entry point [OK]

Hint: API gateway = one door for all client requests [OK]

Common Mistakes:

• Confusing API gateway with database or storage
• Thinking API gateway replaces backend services
• Assuming API gateway manages data directly

2. Which of the following is the correct function of an API gateway in handling client requests?

easy

A. Directly querying the database for each request.

B. Serving static files like images and videos.

C. Routing requests to appropriate backend services.

D. Replacing the client application.

Solution

1. Step 1: Identify API gateway functions

   An API gateway routes incoming client requests to the correct backend services based on the request type or URL.

2. Step 2: Remove unrelated functions

   Serving static files, querying databases directly, or replacing clients are not typical API gateway roles.

3. Final Answer:

   Routing requests to appropriate backend services. -> Option C

4. Quick Check:

   API gateway routes requests = C [OK]

Hint: API gateway directs traffic to backend services [OK]

Common Mistakes:

• Thinking API gateway queries databases directly
• Confusing API gateway with CDN or static server
• Assuming API gateway replaces client apps

3. Consider this simplified API gateway flow:

Client -> API Gateway -> Service A (auth) -> Service B (data)

If the API gateway receives a request and Service A rejects it due to failed authentication, what will the client receive?

medium

A. Authentication error response

B. Data from Service B

C. No response (timeout)

D. Success response with empty data

Solution

1. Step 1: Trace request through services

   The request first goes to Service A for authentication. If Service A rejects it, the request does not proceed further.

2. Step 2: Determine client response

   Since authentication failed, the API gateway returns an authentication error to the client instead of data or success.

3. Final Answer:

   Authentication error response -> Option A

4. Quick Check:

   Failed auth = error response [OK]

Hint: Failed auth stops flow, returns error [OK]

Common Mistakes:

• Assuming data is returned despite failed auth
• Thinking client gets no response (timeout)
• Believing success with empty data is sent

4. An API gateway is configured to route requests to multiple services. However, clients report 404 errors for valid endpoints. What is the most likely cause?

medium

A. Backend services are down.

B. API gateway routing rules are misconfigured.

C. Client requests are malformed.

D. API gateway is overloaded.

Solution

1. Step 1: Analyze 404 error meaning

   A 404 error means the requested resource was not found, often due to incorrect routing or URL mapping.

2. Step 2: Identify likely cause in API gateway

   If backend services are up and clients send correct requests, misconfigured routing rules in the API gateway cause 404 errors.

3. Final Answer:

   API gateway routing rules are misconfigured. -> Option B

4. Quick Check:

   404 = routing misconfiguration [OK]

Hint: 404 usually means wrong routing path [OK]

Common Mistakes:

• Assuming backend services are down without checking
• Blaming client requests without validation
• Thinking overload causes 404 errors

5. You want to design an API gateway that aggregates data from three backend services before responding to the client. Which approach best fits this requirement?

hard

A. API gateway calls all three services, combines their responses, then sends one response to client.

B. API gateway caches data and never calls backend services.

C. API gateway sends separate responses from each service to the client.

D. API gateway forwards client requests directly to one backend service only.

Solution

1. Step 1: Understand aggregation requirement

   The API gateway must collect data from multiple services and combine it into a single response for the client.

2. Step 2: Evaluate options for aggregation

   Forwarding to one service or sending separate responses does not aggregate data. Caching alone does not fulfill aggregation if fresh data is needed.

3. Final Answer:

   API gateway calls all three services, combines their responses, then sends one response to client. -> Option A

4. Quick Check:

   Aggregation = combine multiple service responses [OK]

Hint: Aggregate by combining multiple service responses [OK]

Common Mistakes:

• Thinking forwarding to one service aggregates data
• Sending multiple responses instead of one combined
• Assuming caching replaces aggregation