Why API gateways unify service access in Microservices - Design It to Understand It

Design: API Gateway for Microservices

Design focuses on the API Gateway component and its interaction with microservices and clients. Internal microservice design and database details are out of scope.

Functional Requirements

FR1: Provide a single entry point for all client requests to multiple microservices

FR2: Handle request routing to appropriate microservices based on API paths

FR3: Perform authentication and authorization before forwarding requests

FR4: Aggregate responses from multiple microservices when needed

FR5: Implement rate limiting and request throttling to protect backend services

FR6: Enable protocol translation if clients and services use different protocols

FR7: Support caching of frequent responses to reduce load on microservices

Non-Functional Requirements

NFR1: Must handle 10,000 concurrent client requests

NFR2: API response latency p99 should be under 200ms

NFR3: Availability target of 99.9% uptime (less than 8.77 hours downtime per year)

NFR4: Scalable to add more microservices without changing client configurations

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway server

Authentication and Authorization module

Load balancer

Service registry or discovery

Caching layer

Rate limiter

Logging and monitoring tools

Design Patterns

API Gateway pattern

Backend for Frontend (BFF)

Circuit Breaker for fault tolerance

Request Aggregation

Rate Limiting and Throttling

Service Discovery

Reference Architecture

Client
  |
  v
API Gateway --- Auth Service
  |             |
  |             v
  |          Token Validation
  |
  |---> Service A
  |
  |---> Service B
  |
  |---> Service C
  |
  v
Cache Layer

Logs & Monitoring

Components

API Gateway

Nginx with Lua scripts or Kong API Gateway

Acts as a single entry point, routes requests, enforces policies

Authentication Service

OAuth 2.0 server or JWT validation library

Validates client credentials and tokens

Service Registry

Consul or Eureka

Keeps track of available microservices and their endpoints

Cache Layer

Redis or Memcached

Stores frequent responses to reduce backend load

Rate Limiter

Envoy or custom middleware

Limits request rate to protect backend services

Logging and Monitoring

ELK stack (Elasticsearch, Logstash, Kibana) or Prometheus + Grafana

Tracks request metrics, errors, and system health

Request Flow

1. Client sends request to API Gateway

2. API Gateway authenticates request via Authentication Service

3. If authentication fails, API Gateway returns error to client

4. If authenticated, API Gateway checks rate limits

5. API Gateway routes request to appropriate microservice based on path

6. Microservice processes request and returns response

7. API Gateway caches response if applicable

8. API Gateway aggregates responses if multiple services are involved

9. API Gateway returns final response to client

10. Logs and metrics are recorded for monitoring

Database Schema

Not applicable as API Gateway mainly routes requests and does not store persistent data. Authentication service may have user and token tables, but out of scope here.

Scaling Discussion

Bottlenecks

API Gateway becoming a single point of failure under high load

Authentication service latency affecting overall response time

Cache misses causing increased load on microservices

Rate limiter overhead slowing down request processing

Service registry delays causing routing failures

Solutions

Deploy multiple API Gateway instances behind a load balancer for high availability

Use distributed caching and token validation to reduce auth latency

Implement cache warming and efficient cache invalidation strategies

Optimize rate limiter with in-memory counters and distributed algorithms

Use highly available and fast service discovery mechanisms with health checks

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the API Gateway architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why a single entry point simplifies client interaction

Discuss how API Gateway centralizes cross-cutting concerns like auth and rate limiting

Highlight benefits of response aggregation and protocol translation

Mention importance of caching to improve performance

Address availability and scalability by using multiple gateway instances

Show awareness of potential bottlenecks and mitigation strategies

Practice

1. What is the main purpose of an API gateway in a microservices architecture?

easy

A. To provide a single entry point for clients to access multiple services

B. To store all the data from microservices in one database

C. To replace all microservices with a single monolithic service

D. To directly connect clients to each microservice without any intermediary

Solution

1. Step 1: Understand the role of API gateways

   API gateways act as a single point where clients send requests instead of calling multiple services directly.

2. Step 2: Compare options with this role

   Only To provide a single entry point for clients to access multiple services describes this single entry point role correctly. Other options describe unrelated or incorrect roles.

3. Final Answer:

   To provide a single entry point for clients to access multiple services -> Option A

4. Quick Check:

   API gateway = single entry point [OK]

Hint: API gateway = one door for many services [OK]

Common Mistakes:

• Thinking API gateway stores data
• Confusing API gateway with database
• Believing API gateway replaces microservices

2. Which of the following is a correct function of an API gateway in microservices?

easy

A. Routing client requests to appropriate microservices

B. Directly updating the database of each microservice

C. Replacing the need for load balancers

D. Running all microservices in a single process

Solution

1. Step 1: Identify API gateway responsibilities

   API gateways route client requests to the correct microservice based on the request.

2. Step 2: Evaluate each option

   Routing client requests to appropriate microservices matches routing requests. Options B, C, and D describe unrelated or incorrect functions.

3. Final Answer:

   Routing client requests to appropriate microservices -> Option A

4. Quick Check:

   API gateway routes requests = A [OK]

Hint: API gateway directs traffic like a traffic cop [OK]

Common Mistakes:

• Assuming API gateway updates databases
• Confusing API gateway with load balancer
• Thinking API gateway runs all services

3. Consider this scenario: A client sends a request to an API gateway which then calls three microservices and combines their responses before replying. What is the main benefit of this approach?

medium

A. It increases the number of client requests to microservices

B. It reduces the number of client requests and simplifies client logic

C. It forces clients to handle data merging from multiple services

D. It removes the need for microservices to communicate internally

Solution

1. Step 1: Analyze the API gateway's role in combining responses

   The API gateway collects data from multiple services and sends a single combined response to the client.

2. Step 2: Understand the benefit for the client

   This reduces client complexity and the number of requests it must make, matching It reduces the number of client requests and simplifies client logic.

3. Final Answer:

   It reduces the number of client requests and simplifies client logic -> Option B

4. Quick Check:

   API gateway combines data = fewer client requests [OK]

Hint: API gateway merges data, so client asks less [OK]

Common Mistakes:

• Thinking client must merge data
• Believing requests increase with API gateway
• Assuming microservices stop internal communication

4. A developer notices clients are directly calling multiple microservices, bypassing the API gateway. What issue does this cause?

medium

A. Clients will have fewer requests to manage

B. Microservices will automatically merge responses for clients

C. API gateway will block all client requests

D. Clients lose the benefits of unified access and security provided by the API gateway

Solution

1. Step 1: Understand the role of API gateway in client access

   The API gateway provides unified access and security features like authentication and rate limiting.

2. Step 2: Analyze the effect of bypassing the gateway

   Bypassing removes these benefits, causing clients to handle complexity and security themselves, matching Clients lose the benefits of unified access and security provided by the API gateway.

3. Final Answer:

   Clients lose the benefits of unified access and security provided by the API gateway -> Option D

4. Quick Check:

   Bypassing API gateway = lose unified access [OK]

Hint: Bypass gateway, lose security and simplicity [OK]

Common Mistakes:

• Assuming microservices merge responses automatically
• Thinking API gateway blocks all requests if bypassed
• Believing clients have fewer requests when bypassing

5. In a large system with many microservices, why is using an API gateway better than letting clients call services directly? Choose the best explanation.

hard

A. API gateways replace the need for microservices and store all data centrally

B. Clients can only call one microservice at a time without an API gateway

C. API gateways reduce client complexity, improve security, and enable request routing and response aggregation

D. API gateways slow down requests by adding unnecessary steps

Solution

1. Step 1: List benefits of API gateways in large systems

   They simplify client calls by unifying access, add security layers, route requests properly, and combine responses.

2. Step 2: Compare options with these benefits

   API gateways reduce client complexity, improve security, and enable request routing and response aggregation correctly lists these benefits. Options A, C, and D are incorrect or misleading.

3. Final Answer:

   API gateways reduce client complexity, improve security, and enable request routing and response aggregation -> Option C

4. Quick Check:

   API gateway benefits = B [OK]

Hint: API gateway = simpler, safer, smarter client access [OK]

Common Mistakes:

• Thinking API gateway replaces microservices
• Believing clients cannot call multiple services without gateway
• Assuming API gateway always slows down requests