Bird
Raised Fist0

API Gateway Pattern - Auth, Rate Limiting, Routing, Transformation

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
API Gateway Pattern - Auth, Rate Limiting, Routing, Transformation
Design an API Gateway handling authentication, rate limiting, routing, and payload transformation at scale
mediumAPI Design and GatewayFreq: high
Prerequisites:Token BucketRedis
⚙️
Functional Requirements
  • Authenticate incoming API requests using OAuth 2.0 tokens or API keys.
  • Enforce per-user and per-API rate limiting using token bucket algorithm.
  • Route requests dynamically to appropriate backend microservices based on URL and headers.
  • Transform request and response payloads (e.g., JSON to XML, field masking) as per client or service requirements.
  • Support extensible plugin architecture for adding new policies (e.g., logging, throttling).
  • Provide detailed metrics and logging for monitoring and debugging.
📊
Non-Functional Requirements
  • Handle 2 billion daily active users generating 40 requests per user per day.
  • Achieve 99.99% availability with failover and redundancy.
  • Maintain end-to-end request latency under 100ms P99 including gateway processing.
  • Support peak write QPS of at least 925K requests per second.
  • Scale horizontally with zero downtime deployments.
❓
Clarifying Questions to Ask
  • What authentication methods must the gateway support (OAuth, API keys, JWT)?
  • Are rate limits enforced per user, per API key, or per IP address?
  • What is the expected peak traffic pattern (daily, hourly spikes)?
  • What types of payload transformations are required (format, field masking)?
  • Should the gateway support synchronous and asynchronous request handling?
  • Is TLS termination handled at the gateway or upstream?
Out of Scope
✗ Design of backend microservices beyond routing.
✗ User identity management and token issuance (assumed external).
✗ Complex API versioning and backward compatibility strategies.
✗ Deep payload validation or business logic enforcement.
📌
Assumptions
  • 2 billion daily active users (DAU).
  • Each user generates 40 API requests per day on average.
  • Average request payload size is 100 bytes.
  • Retention period for logs and metrics is 30 days.
⚡
QPS Calculation
write qps2,000,000,000 × 40 / 86400 ≈ 925,926 requests/sec
peak write qpsAssuming 2× peak factor: 925,926 × 2 = 1,851,852 requests/sec
read qpsAssuming 3× read amplification for auth cache and metrics: 925,926 × 3 ≈ 2,777,778 requests/sec
💾
Storage Estimate
daily storage925,926 req/sec × 100 bytes × 86400 sec ≈ 8 TB/day
retention storage8 TB/day × 30 days = 240 TB
with replication240 TB × 3 replicas = 720 TB total storage
🖥️
Server Estimate
auth serversAssuming 10,000 req/sec per auth server: 1,851,852 / 10,000 ≈ 186 servers
rate limit redisRedis cluster with 500K ops/sec per shard: 1,851,852 × 2 (read+write) / 500,000 ≈ 8 shards
🏗️
Architecture Diagram
Architecture Diagram
Client
  |
  v
API Gateway Load Balancer
  |
  v
API Gateway Nodes
  |  |  |
  |  |  +--> Redis Cluster (Rate Limiting)
  |  +----> Auth Service (OAuth/JWT Validation)
  +-------> Backend Services (Microservices)
           |
           v
       Metrics & Logging
🧩
Components
API Gateway Nodes
Core request processing: authentication, rate limiting, routing, transformation
Why thisEnvoy proxy with custom filters supports high throughput, pluggable filters for auth, rate limiting, and transformation with sub-10ms overhead
Why notNginx lacks native support for dynamic routing and complex payload transformation at scale without heavy custom modules
Redis Cluster
Store token bucket counters for rate limiting with low latency
Why thisRedis supports atomic INCR and EXPIRE commands with sub-millisecond latency and native TTL for token bucket expiry
Why notMemcached lacks persistence and atomic counters, making rate limiting inconsistent under failures
Auth Service
Validate OAuth tokens and API keys, provide user identity info
Why thisDedicated OAuth 2.0 compliant service with caching layer to offload gateway and reduce latency
Why notEmbedding auth logic in gateway increases complexity and reduces maintainability
API Gateway Load Balancer
Distribute incoming requests evenly across gateway nodes
Why thisAWS ALB or Envoy-based L4 load balancer supports TLS termination and health checks with low latency
Why notDNS round robin lacks health checks and fine-grained load balancing, causing hotspots
Metrics & Logging Pipeline
Collect request logs, latency metrics, and rate limit counters for monitoring
Why thisKafka-based pipeline enables asynchronous, durable, and scalable log ingestion with real-time analytics
Why notDirect DB writes cause high latency and backpressure on gateway nodes
🔄
Data Flow
1
Client → API Gateway Load Balancer
Send API request over HTTPS
TLS termination at load balancer reduces gateway CPU load
2
Load Balancer → API Gateway Node
Forward request to healthy gateway node
Load balancer performs health checks and distributes load evenly
3
API Gateway Node → Auth Service
Validate OAuth token or API key
Auth service caches tokens to reduce latency
4
API Gateway Node → Redis Cluster
Check and update token bucket counters for rate limiting
Atomic Redis commands ensure consistent rate limiting under concurrency
5
API Gateway Node → Backend Service
Route request based on URL and headers
Dynamic routing supports multiple microservices
6
API Gateway Node → Client
Transform response payload and send back
Payload transformation supports client-specific formats
7
API Gateway Node → Metrics & Logging Pipeline
Emit logs and metrics asynchronously
Non-blocking to avoid adding latency

Practice

(1/5)
1. A social media platform needs to store user posts with high write throughput and ensure efficient retrieval by user and timestamp. Which component best handles this workload?
easy
A. A wide-column store like Cassandra with partition keys on user ID
B. A relational database with normalized tables and joins
C. A document store like MongoDB with embedded user posts
D. An in-memory cache like Redis without persistence

Solution

  1. Step 1: Identify write throughput and query pattern

    The system requires high write throughput and efficient retrieval by user and timestamp, which suits wide-column stores that partition data by user ID.

  2. Step 2: Evaluate options against constraints

    Relational DBs struggle with high write throughput and joins; document stores lack efficient range queries by timestamp; Redis lacks durability and is not ideal for large persistent datasets.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Wide-column store partitions by user ID -> high write throughput and efficient user queries [check]
Hint: Partition by user ID for high write and read throughput [check]
Common Mistakes:
2. Without the automated rollback system, what exactly breaks when a misconfiguration causes a route leak?
medium
A. The system silently drops all BGP updates, causing network partition
B. Route leak detection alerts flood the monitoring system but traffic continues normally
C. The system automatically quarantines all routes, causing a complete service outage
D. Alice's traffic is routed incorrectly; operators must manually detect and fix the leak, causing prolonged outages

Solution

  1. Step 1: Trace failure without rollback

    Misconfigurations propagate incorrect routes; without rollback, manual detection delays fix.

  2. Step 2: Eliminate other failure modes

    Silent drops (A) don't occur; automatic quarantine (C) would cause total outage, not typical; alert floods (D) don't prevent traffic misrouting.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Manual fix delays cause prolonged incorrect routing [check]
Hint: No rollback means manual fix delays outages [check]
Common Mistakes:
3. For a distributed cache with frequent node joins and leaves, which approach best balances minimal key remapping and low client complexity?
medium
A. Use a centralized directory service to map keys to nodes
B. Use consistent hashing with virtual nodes and client-side slot caching
C. Use simple modulo hashing with client retries on failures
D. Use random key assignment with periodic full cache rebuilds

Solution

  1. Step 1: Evaluate centralized directory approach

    Centralized directory (Use a centralized directory service to map keys to nodes) adds single point of failure and latency, increasing client complexity.

  2. Step 2: Compare consistent hashing with virtual nodes

    Consistent hashing with virtual nodes and client-side caching (Use consistent hashing with virtual nodes and client-side slot caching) minimizes remapping and keeps client complexity manageable.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Consistent hashing + virtual nodes + client caching -> minimal remap, manageable complexity [check]
Hint: Consistent hashing + virtual nodes + client caching balances remap and complexity [OK]
Common Mistakes:
4. For a global API rate limiter with strict per-user limits, which architecture best balances consistency and latency?
medium
A. Use client-side counters with periodic server reconciliation
B. Use a single centralized Redis instance for all counters
C. Use a sharded Redis cluster with consistent hashing per user ID
D. Use a distributed SQL database with strong consistency guarantees

Solution

  1. Step 1: Analyze consistency and latency needs

    Strict per-user limits require consistent counters with low latency.

  2. Step 2: Evaluate architectures

    Centralized Redis (A) is a bottleneck; client-side counters (C) are eventually consistent and unreliable; distributed SQL (D) has high latency. Sharded Redis with consistent hashing (B) balances load and maintains per-user consistency.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Sharded Redis cluster scales with consistent hashing per user [check]
Hint: Shard counters by user ID for scalable consistency [check]
Common Mistakes:
5. If the Kafka cluster responsible for vote event streaming goes down, what exactly breaks in the Reddit voting system?
medium
A. Vote events cannot be buffered, causing vote submission requests to block or fail immediately
B. The UI shows stale post scores but votes are queued in Redis for later processing
C. Votes continue to be processed but ranking updates become inconsistent
D. Users can still vote, but votes are lost silently without any retry

Solution

  1. Step 1: Understand Kafka's role in buffering vote events

    Kafka buffers vote events reliably; if it fails, vote events cannot be streamed to processors.

  2. Step 2: Analyze failure impact

    Vote submissions do not block immediately (Vote events cannot be buffered, causing vote submission requests to block or fail immediately wrong) because frontends can still accept votes and cache temporarily. Votes are not lost silently (Users can still vote, but votes are lost silently without any retry wrong) because Redis or other caches buffer them. UI shows stale scores due to lack of processed updates (The UI shows stale post scores but votes are queued in Redis for later processing correct). Votes continue to be processed but ranking updates become inconsistent is incorrect as votes stop flowing, not just ranking inconsistency.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Kafka failure causes stale UI scores; votes buffered in Redis [check]
Hint: Kafka down -> stale UI, votes buffered temporarily [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard