Why API gateways unify service access in Microservices - Scalability Evidence

Scalability Analysis - Why API gateways unify service access

Growth Table: User Scale and System Changes

Users	System Changes
100 users	Direct service calls; simple routing; minimal latency; no gateway needed
10,000 users	Multiple microservices; need unified access; increased request volume; API gateway introduced for routing and security
1,000,000 users	High concurrency; gateway handles load balancing, authentication, rate limiting; caching added; gateway scales horizontally
100,000,000 users	Global distribution; multiple API gateway clusters; CDN integration; advanced traffic shaping; microservices sharded; gateway handles failover and analytics

First Bottleneck: API Gateway Throughput and Latency

As user requests grow, the API gateway becomes the first bottleneck because it handles all incoming traffic to multiple microservices. It must route, authenticate, and apply policies for every request. Without scaling, the gateway's CPU, memory, or network bandwidth limits will cause increased latency and dropped requests.

Scaling Solutions for API Gateway Bottleneck

• Horizontal Scaling: Add more gateway instances behind a load balancer to distribute traffic.
• Caching: Cache common responses at the gateway to reduce backend calls.
• Rate Limiting: Protect backend services by limiting requests per user or IP.
• Edge Deployment: Deploy gateways closer to users (regional clusters) to reduce latency.
• Offload SSL/TLS: Terminate encryption at the gateway to reduce backend load.
• Use CDN: For static content, reduce gateway load by serving from CDN.

Back-of-Envelope Cost Analysis

• At 1M users, assuming 1 request per second each = 1M RPS total.
• One gateway instance handles ~5,000 RPS → need ~200 instances.
• Network bandwidth per gateway: 1 Gbps (~125 MB/s) can handle ~10,000 requests of 10 KB each per second.
• Storage at gateway is minimal (caching few GBs), but backend storage grows with data.
• Cost scales with number of gateway instances, bandwidth, and caching infrastructure.

Interview Tip: Structuring Scalability Discussion

Start by explaining the role of the API gateway in unifying access. Discuss how traffic growth impacts the gateway first. Identify bottlenecks like CPU, memory, and network. Propose scaling solutions like horizontal scaling and caching. Mention trade-offs and monitoring needs. Use clear examples and numbers to support your points.

Self Check Question

Your API gateway handles 1,000 requests per second. Traffic grows 10x to 10,000 RPS. What do you do first and why?

Key Result

API gateways unify access but become the first bottleneck as traffic grows; horizontal scaling and caching at the gateway are key to handle millions of users efficiently.

Practice

1. What is the main purpose of an API gateway in a microservices architecture?

easy

A. To provide a single entry point for clients to access multiple services

B. To store all the data from microservices in one database

C. To replace all microservices with a single monolithic service

D. To directly connect clients to each microservice without any intermediary

Solution

1. Step 1: Understand the role of API gateways

   API gateways act as a single point where clients send requests instead of calling multiple services directly.

2. Step 2: Compare options with this role

   Only To provide a single entry point for clients to access multiple services describes this single entry point role correctly. Other options describe unrelated or incorrect roles.

3. Final Answer:

   To provide a single entry point for clients to access multiple services -> Option A

4. Quick Check:

   API gateway = single entry point [OK]

Hint: API gateway = one door for many services [OK]

Common Mistakes:

• Thinking API gateway stores data
• Confusing API gateway with database
• Believing API gateway replaces microservices

2. Which of the following is a correct function of an API gateway in microservices?

easy

A. Routing client requests to appropriate microservices

B. Directly updating the database of each microservice

C. Replacing the need for load balancers

D. Running all microservices in a single process

Solution

1. Step 1: Identify API gateway responsibilities

   API gateways route client requests to the correct microservice based on the request.

2. Step 2: Evaluate each option

   Routing client requests to appropriate microservices matches routing requests. Options B, C, and D describe unrelated or incorrect functions.

3. Final Answer:

   Routing client requests to appropriate microservices -> Option A

4. Quick Check:

   API gateway routes requests = A [OK]

Hint: API gateway directs traffic like a traffic cop [OK]

Common Mistakes:

• Assuming API gateway updates databases
• Confusing API gateway with load balancer
• Thinking API gateway runs all services

3. Consider this scenario: A client sends a request to an API gateway which then calls three microservices and combines their responses before replying. What is the main benefit of this approach?

medium

A. It increases the number of client requests to microservices

B. It reduces the number of client requests and simplifies client logic

C. It forces clients to handle data merging from multiple services

D. It removes the need for microservices to communicate internally

Solution

1. Step 1: Analyze the API gateway's role in combining responses

   The API gateway collects data from multiple services and sends a single combined response to the client.

2. Step 2: Understand the benefit for the client

   This reduces client complexity and the number of requests it must make, matching It reduces the number of client requests and simplifies client logic.

3. Final Answer:

   It reduces the number of client requests and simplifies client logic -> Option B

4. Quick Check:

   API gateway combines data = fewer client requests [OK]

Hint: API gateway merges data, so client asks less [OK]

Common Mistakes:

• Thinking client must merge data
• Believing requests increase with API gateway
• Assuming microservices stop internal communication

4. A developer notices clients are directly calling multiple microservices, bypassing the API gateway. What issue does this cause?

medium

A. Clients will have fewer requests to manage

B. Microservices will automatically merge responses for clients

C. API gateway will block all client requests

D. Clients lose the benefits of unified access and security provided by the API gateway

Solution

1. Step 1: Understand the role of API gateway in client access

   The API gateway provides unified access and security features like authentication and rate limiting.

2. Step 2: Analyze the effect of bypassing the gateway

   Bypassing removes these benefits, causing clients to handle complexity and security themselves, matching Clients lose the benefits of unified access and security provided by the API gateway.

3. Final Answer:

   Clients lose the benefits of unified access and security provided by the API gateway -> Option D

4. Quick Check:

   Bypassing API gateway = lose unified access [OK]

Hint: Bypass gateway, lose security and simplicity [OK]

Common Mistakes:

• Assuming microservices merge responses automatically
• Thinking API gateway blocks all requests if bypassed
• Believing clients have fewer requests when bypassing

5. In a large system with many microservices, why is using an API gateway better than letting clients call services directly? Choose the best explanation.

hard

A. API gateways replace the need for microservices and store all data centrally

B. Clients can only call one microservice at a time without an API gateway

C. API gateways reduce client complexity, improve security, and enable request routing and response aggregation

D. API gateways slow down requests by adding unnecessary steps

Solution

1. Step 1: List benefits of API gateways in large systems

   They simplify client calls by unifying access, add security layers, route requests properly, and combine responses.

2. Step 2: Compare options with these benefits

   API gateways reduce client complexity, improve security, and enable request routing and response aggregation correctly lists these benefits. Options A, C, and D are incorrect or misleading.

3. Final Answer:

   API gateways reduce client complexity, improve security, and enable request routing and response aggregation -> Option C

4. Quick Check:

   API gateway benefits = B [OK]

Hint: API gateway = simpler, safer, smarter client access [OK]

Common Mistakes:

• Thinking API gateway replaces microservices
• Believing clients cannot call multiple services without gateway
• Assuming API gateway always slows down requests