Bird
Raised Fist0

API Gateway Pattern - Auth, Rate Limiting, Routing, Transformation

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
API Gateway Pattern - Auth, Rate Limiting, Routing, Transformation
45 min · senior
senior

Design an API Gateway for a 50 million daily active user (DAU) startup that handles authentication, rate limiting, request routing, and payload transformation. The system must support encrypted traffic end-to-end, enforce strict rate limits per user and API key, route requests to multiple backend microservices, and transform request/response payloads for backward compatibility. The P99 latency must be under 200ms to maintain a responsive user experience.

Constraints
  • End-to-end encryption (TLS) mandatory for all client-to-gateway and gateway-to-backend communication
  • Rate limiting must support burst and sustained limits per user and API key
  • P99 latency under 200ms including auth, rate limiting, routing, and transformation
  • Support dynamic routing rules to multiple backend microservices with failover
  • Payload transformation must support JSON schema versioning and backward compatibility
📋
Guided Steps
1/5
Step 1
Authentication Mechanism
How will you design the authentication flow in the API Gateway to securely identify clients?
💡 Hint
Consider token-based authentication with stateless validation to minimize latency.
Key Decision
Use JWT tokens signed with RSA keys validated by the API Gateway locally to avoid network calls for auth verification.
Interviewer Expectation
Mid: Names JWT for stateless auth. Senior: Explains RSA signature validation locally avoids auth DB calls, reducing latency and scaling to 50M DAU.
⚠️Common Wrong Answer
Using session-based authentication with centralized DB lookups causes high latency and DB bottlenecks at scale.
Step 2
Rate Limiting Strategy
How will you implement rate limiting per user and API key to enforce burst and sustained limits?
💡 Hint
Think about distributed counters with low latency and atomicity guarantees.
Key Decision
Use Redis with Lua scripts for atomic token bucket rate limiting per user and API key, deployed as a Redis Cluster for horizontal scaling.
Interviewer Expectation
Mid: Mentions Redis for rate limiting. Senior: Details Lua scripts for atomicity, Redis Cluster for scale, and separate keys for burst vs sustained limits.
⚠️Common Wrong Answer
Using in-memory counters on API Gateway nodes leads to inconsistent limits due to lack of synchronization.
Step 3
Routing Requests to Backend Services
How will the API Gateway route requests to multiple backend microservices with failover support?
💡 Hint
Consider service discovery and load balancing integrated with health checks.
Key Decision
Use Envoy proxy as the API Gateway with dynamic routing configured via xDS APIs connected to a service registry like Consul for health-aware routing and failover.
Interviewer Expectation
Mid: Names Envoy for routing. Senior: Explains dynamic xDS config, Consul integration for health checks, and failover routing to avoid downtime.
⚠️Common Wrong Answer
Hardcoding backend IPs in the gateway config causes downtime and manual intervention on service changes.
Step 4
Payload Transformation
How will you implement request and response payload transformation to support JSON schema versioning and backward compatibility?
💡 Hint
Think about a transformation layer that can apply schema mappings dynamically.
Key Decision
Integrate a transformation filter in Envoy using WASM modules to apply JSON schema transformations at runtime without backend changes.
Interviewer Expectation
Mid: Suggests transformation filters. Senior: Details WASM modules in Envoy for low-latency, dynamic transformations supporting multiple schema versions.
⚠️Common Wrong Answer
Performing transformations in backend services increases complexity and latency, breaking separation of concerns.
Step 5
Ensuring Low Latency and High Availability
How will you ensure the API Gateway meets the P99 latency < 200ms SLA and remains highly available?
💡 Hint
Consider horizontal scaling, caching, and circuit breakers.
Key Decision
Deploy Envoy API Gateway in multiple availability zones behind an AWS ALB with Redis Cluster for caching auth and rate limit state, plus circuit breakers to degrade gracefully.
Interviewer Expectation
Mid: Mentions horizontal scaling and caching. Senior: Explains multi-AZ deployment, Redis caching for auth tokens and rate limits, and circuit breakers to prevent cascading failures.
⚠️Common Wrong Answer
Single instance gateway or no caching leads to high latency and single points of failure.
🔌
API Design Challenge
Sketch 3 core API Gateway endpoints to handle client requests: authentication, rate-limited request forwarding, and response transformation.
Hints
  • How does the client send authentication credentials and receive tokens?
  • How does the gateway enforce rate limits before forwarding requests?
  • How does the gateway transform backend responses before returning to clients?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Stateless JWT authentication with local RSA signature validation to scale to 50M DAU
  • Distributed rate limiting using Redis Cluster with atomic Lua scripts for burst and sustained limits
  • Dynamic routing with Envoy proxy integrated with Consul for health checks and failover
  • Payload transformation implemented via Envoy WASM filters for low latency and backward compatibility
Good to Have - 30%
  • Multi-AZ deployment behind AWS ALB for high availability and horizontal scaling
  • Caching of auth tokens and rate limit counters in Redis to reduce latency
  • Circuit breakers to prevent cascading failures and maintain SLA
Bonus - 10%
  • Detailed API design sketch with authentication, rate-limited request forwarding, and response transformation endpoints
  • Explanation of TLS termination and re-encryption strategy for end-to-end encryption
Anti-patterns to Avoid
Session-based authentication with centralized DB lookups causes high latency and DB bottlenecks at scale, leading to poor user experience and system overload
In-memory rate limiting counters on API Gateway nodes lead to inconsistent enforcement under scale due to lack of synchronization, causing unfair throttling or abuse
Hardcoded backend IPs in gateway config cause downtime and manual intervention on service changes, reducing system resilience and increasing operational overhead
Performing payload transformations in backend services increases latency and coupling, breaking separation of concerns and complicating service maintenance
Single instance API Gateway causes single point of failure and inability to scale, risking system outages and degraded performance
No caching of auth or rate limit state causes repeated expensive lookups and SLA violations, increasing latency and backend load
✅
Model Answer
▼ Reveal

The API Gateway design uses stateless JWT authentication with RSA signature validation locally to minimize latency and avoid DB calls. Rate limiting is implemented using Redis Cluster with Lua scripts for atomic token bucket algorithms supporting burst and sustained limits per user and API key. Envoy proxy serves as the API Gateway with dynamic routing configured via xDS APIs connected to Consul for health-aware routing and failover. Payload transformation is handled by WASM-based filters in Envoy to support JSON schema versioning and backward compatibility. The system is deployed multi-AZ behind an AWS ALB for high availability and horizontal scaling, with Redis caching for auth tokens and rate limit counters to maintain P99 latency under 200ms. Circuit breakers in Envoy detect backend failures and degrade gracefully. TLS termination and re-encryption at the gateway enforce end-to-end encryption.

Key Decisions
  • Use JWT tokens signed with RSA keys validated locally in the API Gateway to avoid network calls and reduce latency
  • Implement rate limiting using Redis Cluster with Lua scripts for atomic token bucket algorithms per user and API key
  • Employ Envoy proxy with dynamic routing via xDS APIs connected to Consul for health-aware routing and failover
  • Integrate WASM-based transformation filters in Envoy to apply JSON schema transformations at runtime without backend changes
  • Deploy Envoy API Gateway in multiple availability zones behind AWS ALB for high availability and horizontal scaling
  • Cache authentication tokens and rate limit counters in Redis to reduce repeated lookups and maintain SLA
  • Use circuit breakers in Envoy to detect backend failures and degrade gracefully
  • Enforce end-to-end encryption with TLS termination and re-encryption at the API Gateway
  • JWT tokens with RSA signature validation locally in API Gateway to avoid auth DB calls and reduce latency
  • Redis Cluster with Lua scripts for atomic token bucket rate limiting per user and API key to handle burst and sustained limits
  • Envoy proxy as API Gateway with dynamic routing via xDS APIs connected to Consul service registry for health-aware routing and failover
  • WASM-based transformation filters in Envoy to apply JSON schema versioning and backward-compatible payload transformations at runtime
  • Multi-AZ deployment of Envoy behind AWS Application Load Balancer for high availability and horizontal scaling
  • Redis caching of authentication tokens and rate limit counters to reduce repeated lookups and maintain P99 latency under 200ms
  • Circuit breakers in Envoy to detect backend failures and degrade gracefully without impacting client experience
  • TLS termination and re-encryption at API Gateway to enforce end-to-end encryption between client and backend services

Practice

(1/5)
1. In the driver matching flow, at step 4 the Matching Service queries the Surge Pricing Service for the current multiplier. What happens if the Surge Pricing Service responds with a stale multiplier due to cache lag?
easy
A. The Matching Service rejects the request and retries immediately
B. The Matching Service blocks until fresh surge data is available
C. The Matching Service falls back to a default multiplier of 1.0
D. The Matching Service uses the stale multiplier, causing temporary incorrect pricing

Solution

  1. Step 1: Trace Matching Service behavior at step 4

    Matching Service queries Surge Pricing Service for multiplier to calculate fare.

  2. Step 2: Understand stale data impact

    If surge multiplier is stale due to cache lag, Matching Service proceeds with returned value, causing temporary fare mispricing but no blocking or retries.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Matching Service tolerates stale surge multiplier for availability [check]
Hint: Matching tolerates stale surge data to avoid blocking [check]
Common Mistakes:
2. For a system requiring both low latency and strong consistency in user profile updates, which architecture choice best balances these constraints?
medium
A. Use eventual consistency with asynchronous replication to maximize availability
B. Use a single primary database with synchronous replication for strong consistency
C. Use a distributed cache with write-behind to the database for low latency
D. Use client-side caching to reduce read latency without server coordination

Solution

  1. Step 1: Identify consistency and latency requirements

    Strong consistency requires synchronous replication; low latency demands minimal delay.

  2. Step 2: Evaluate architecture options

    Single primary DB with sync replication ensures consistency but may increase latency slightly; best trade-off.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Synchronous replication balances strong consistency and latency [check]
Hint: Strong consistency needs sync replication, not eventual or cache-only [check]
Common Mistakes:
3. If the Key Management Service (KMS) becomes unavailable, what exactly breaks in the system's encryption-at-rest workflow?
medium
A. New encryption keys cannot be generated, but existing data remains accessible and decryptable
B. Data encryption stops, causing all writes to fail immediately
C. Decryption of existing data fails, causing read operations to block or error out
D. The system falls back to unencrypted storage, risking data exposure

Solution

  1. Step 1: Understand KMS role in key lifecycle

    KMS generates and manages encryption keys; if unavailable, new keys can't be created but existing keys cached locally allow decryption.

  2. Step 2: Eliminate incorrect failure modes

    Encryption doesn't stop immediately; decryption continues with cached keys; system does not fallback to unencrypted storage automatically.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    KMS outage blocks new key generation, existing keys still usable [check]
Hint: KMS down -> no new keys, existing keys still decrypt [check]
Common Mistakes:
4. For a high-throughput e-commerce system requiring strong consistency and complex joins, which database choice best balances performance and transactional integrity?
medium
A. Use a sharded relational database with two-phase commit for distributed transactions
B. Use an in-memory cache with asynchronous writes to a relational DB
C. Use a single-node relational database with vertical scaling
D. Use a distributed NoSQL key-value store with eventual consistency

Solution

  1. Step 1: Analyze requirements

    Strong consistency and complex joins require relational capabilities and transactional integrity across shards.

  2. Step 2: Evaluate options

    Sharded relational DB with two-phase commit supports distributed transactions and joins, unlike NoSQL or cache-based approaches.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Sharded RDB + 2PC balances scale and ACID [check]
Hint: Distributed transactions need 2PC, not eventual consistency stores [check]
Common Mistakes:
5. At 10x scale, which component in a GraphQL API gateway is most likely to become the bottleneck and why?
hard
A. The underlying database, because it cannot handle 10x query volume
B. The GraphQL query parsing and validation layer, due to CPU-intensive query complexity checks on every request
C. The network load balancer, because it cannot distribute traffic evenly at high scale
D. The client SDK, because it caches too much data locally

Solution

  1. Step 1: Identify components stressed by 10x scale

    Database scaling is often handled by sharding or replicas; load balancers scale horizontally; client SDKs are local.

  2. Step 2: Recognize GraphQL parsing and validation cost

    Parsing and complexity validation are CPU-bound and happen per request, becoming CPU bottlenecks at high request rates.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    GraphQL validation CPU cost scales linearly with requests [check]
Hint: GraphQL validation CPU cost limits scale first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard