Why API gateways unify service access in Microservices - Design It to Understand It

Design: API Gateway for Microservices

Design focuses on the API Gateway component and its interaction with microservices and clients. Internal microservice design and database details are out of scope.

Functional Requirements

FR1: Provide a single entry point for all client requests to multiple microservices

FR2: Handle request routing to appropriate microservices based on API paths

FR3: Perform authentication and authorization before forwarding requests

FR4: Aggregate responses from multiple microservices when needed

FR5: Implement rate limiting and request throttling to protect backend services

FR6: Enable protocol translation if clients and services use different protocols

FR7: Support caching of frequent responses to reduce load on microservices

Non-Functional Requirements

NFR1: Must handle 10,000 concurrent client requests

NFR2: API response latency p99 should be under 200ms

NFR3: Availability target of 99.9% uptime (less than 8.77 hours downtime per year)

NFR4: Scalable to add more microservices without changing client configurations

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway server

Authentication and Authorization module

Load balancer

Service registry or discovery

Caching layer

Rate limiter

Logging and monitoring tools

Design Patterns

API Gateway pattern

Backend for Frontend (BFF)

Circuit Breaker for fault tolerance

Request Aggregation

Rate Limiting and Throttling

Service Discovery

Reference Architecture

Client
  |
  v
API Gateway --- Auth Service
  |             |
  |             v
  |          Token Validation
  |
  |---> Service A
  |
  |---> Service B
  |
  |---> Service C
  |
  v
Cache Layer

Logs & Monitoring

Components

API Gateway

Nginx with Lua scripts or Kong API Gateway

Acts as a single entry point, routes requests, enforces policies

Authentication Service

OAuth 2.0 server or JWT validation library

Validates client credentials and tokens

Service Registry

Consul or Eureka

Keeps track of available microservices and their endpoints

Cache Layer

Redis or Memcached

Stores frequent responses to reduce backend load

Rate Limiter

Envoy or custom middleware

Limits request rate to protect backend services

Logging and Monitoring

ELK stack (Elasticsearch, Logstash, Kibana) or Prometheus + Grafana

Tracks request metrics, errors, and system health

Request Flow

1. Client sends request to API Gateway

2. API Gateway authenticates request via Authentication Service

3. If authentication fails, API Gateway returns error to client

4. If authenticated, API Gateway checks rate limits

5. API Gateway routes request to appropriate microservice based on path

6. Microservice processes request and returns response

7. API Gateway caches response if applicable

8. API Gateway aggregates responses if multiple services are involved

9. API Gateway returns final response to client

10. Logs and metrics are recorded for monitoring

Database Schema

Not applicable as API Gateway mainly routes requests and does not store persistent data. Authentication service may have user and token tables, but out of scope here.

Scaling Discussion

Bottlenecks

API Gateway becoming a single point of failure under high load

Authentication service latency affecting overall response time

Cache misses causing increased load on microservices

Rate limiter overhead slowing down request processing

Service registry delays causing routing failures

Solutions

Deploy multiple API Gateway instances behind a load balancer for high availability

Use distributed caching and token validation to reduce auth latency

Implement cache warming and efficient cache invalidation strategies

Optimize rate limiter with in-memory counters and distributed algorithms

Use highly available and fast service discovery mechanisms with health checks

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing the API Gateway architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain why a single entry point simplifies client interaction

Discuss how API Gateway centralizes cross-cutting concerns like auth and rate limiting

Highlight benefits of response aggregation and protocol translation

Mention importance of caching to improve performance

Address availability and scalability by using multiple gateway instances

Show awareness of potential bottlenecks and mitigation strategies