Popular gateways (Kong, AWS API Gateway, Nginx) in Microservices - Scalability & System Analysis

Scalability Analysis - Popular gateways (Kong, AWS API Gateway, Nginx)

Growth Table: Popular API Gateways

Users / Traffic	What Changes?	Gateway Load	Latency Impact	Security & Features
100 users	Basic routing, simple auth	Single gateway instance handles traffic	Low latency, minimal overhead	Basic rate limiting, logging
10,000 users	Increased requests, more auth checks	Multiple gateway instances behind load balancer	Latency slightly increases due to processing	Advanced rate limiting, caching enabled
1,000,000 users	High concurrency, complex routing rules	Horizontal scaling of gateways, caching layers added	Latency managed with caching and optimized configs	Security policies, JWT validation, throttling
100,000,000 users	Massive traffic, global distribution	Multi-region gateway clusters, CDN integration	Latency minimized via edge caching and CDNs	WAF, DDoS protection, advanced analytics

First Bottleneck

The first bottleneck is usually the API gateway server CPU and memory. As traffic grows, the gateway must process authentication, routing, rate limiting, and logging for every request. This processing can overwhelm a single instance, causing increased latency and dropped requests.

Scaling Solutions

• Horizontal Scaling: Add more gateway instances behind a load balancer to distribute traffic.
• Caching: Use response caching to reduce repeated processing for the same requests.
• Rate Limiting: Protect backend services by limiting requests per user or IP.
• Offload SSL/TLS: Terminate SSL at load balancer or CDN to reduce gateway CPU load.
• Use CDN: For static content and some API responses, reduce load on gateways.
• Sharding: Route traffic based on user segments or regions to different gateway clusters.

Back-of-Envelope Cost Analysis

• At 1 million users, assuming 1 request per second each, gateways handle ~1 million RPS.
• A single gateway instance handles ~3000 RPS; need ~350 instances for 1M RPS.
• Storage for logs: 1M RPS * 100 bytes/log * 3600 seconds = ~360 GB/hour.
• Bandwidth: 1M RPS * 1 KB/request = ~1 GB/s (~8 Gbps network).
• Costs rise with instances, bandwidth, and storage for logs and metrics.

Interview Tip

Start by identifying the main components and their limits. Discuss how traffic growth affects each part, especially the gateway. Then propose clear scaling steps: horizontal scaling, caching, and offloading work. Always mention trade-offs like cost and complexity.

Self Check

Your API gateway handles 3000 requests per second. Traffic grows 10x to 30,000 RPS. What do you do first?

Answer: Add more gateway instances behind a load balancer to distribute the load horizontally. This prevents CPU and memory overload on a single instance and maintains low latency.

Key Result

API gateways first hit CPU/memory limits as traffic grows; horizontal scaling and caching are key to handle millions of requests per second.

Practice

1. Which of the following is a primary role of API gateways like Kong, AWS API Gateway, or Nginx in microservices?

easy

A. Control and protect communication between services

B. Store large amounts of data

C. Run backend business logic

D. Replace databases in microservices

Solution

1. Step 1: Understand the role of API gateways

   API gateways act as a control point for requests between clients and microservices, managing traffic and security.

2. Step 2: Compare options with gateway functions

   Storing data, running business logic, or replacing databases are not typical gateway roles.

3. Final Answer:

   Control and protect communication between services -> Option A

4. Quick Check:

   Gateway role = Control communication [OK]

Hint: Gateways manage traffic and security, not data storage [OK]

Common Mistakes:

• Confusing gateways with databases
• Thinking gateways run business logic
• Assuming gateways store data

2. Which syntax correctly defines a route in Kong's configuration to forward requests to a service?

easy

A. routes:\n - name example-route\n path: '/example'\n service: example-service

B. routes:\n - name: example-route\n paths: ['/example']\n service: example-service

C. routes:\n - name: example-route\n paths: '/example'\n service: example-service

D. routes:\n - example-route:\n paths: ['/example']\n service: example-service

Solution

1. Step 1: Review Kong route syntax

   Kong routes use a list with keys: name, paths (as a list), and service.

2. Step 2: Identify correct YAML structure

   routes:\n - name: example-route\n paths: ['/example']\n service: example-service correctly uses a list with dash, keys with colons, and paths as a list.

3. Final Answer:

   routes:\n - name: example-route\n paths: ['/example']\n service: example-service -> Option B

4. Quick Check:

   Kong route syntax = routes:\n - name: example-route\n paths: ['/example']\n service: example-service [OK]

Hint: YAML lists need dashes and keys with colons [OK]

Common Mistakes:

• Missing colon after keys
• Using string instead of list for paths
• Incorrect indentation or dash placement

3. Given this Nginx configuration snippet, what happens when a client requests /api/users?

location /api/ {
  proxy_pass http://backend-service/;
}

medium

A. The request is forwarded to http://backend-service/users

B. The request returns a 404 error

C. The request is blocked by Nginx

D. The request is forwarded to http://backend-service/api/users

Solution

1. Step 1: Understand Nginx proxy_pass behavior with trailing slash

   When proxy_pass URL ends with a slash, Nginx replaces the matching location prefix with the proxy URL path.

2. Step 2: Apply to given example

   Location prefix is /api/, proxy_pass is http://backend-service/, so /api/ is replaced by /, forwarding /users to backend-service.

3. Final Answer:

   The request is forwarded to http://backend-service/users -> Option A

4. Quick Check:

   Trailing slash in proxy_pass removes location prefix [OK]

Hint: Trailing slash in proxy_pass removes location prefix [OK]

Common Mistakes:

• Assuming full path is appended
• Confusing proxy_pass with or without trailing slash
• Thinking request is blocked or 404

4. You configured AWS API Gateway with a resource path /items and a GET method, but requests to /items return 403 Forbidden. What is the most likely cause?

medium

A. The backend service URL is incorrect

B. The API Gateway does not support GET methods

C. The GET method is not deployed or enabled in the stage

D. The client IP is blocked by AWS firewall

Solution

1. Step 1: Check AWS API Gateway method deployment

   Methods must be deployed and enabled in the stage to accept requests.

2. Step 2: Understand 403 Forbidden meaning in API Gateway

   403 often means method exists but is not authorized or deployed, not backend URL or IP block.

3. Final Answer:

   The GET method is not deployed or enabled in the stage -> Option C

4. Quick Check:

   403 = method not deployed/enabled [OK]

Hint: Deploy methods in stage to avoid 403 errors [OK]

Common Mistakes:

• Assuming backend URL causes 403
• Thinking API Gateway disallows GET
• Blaming client IP blocking without evidence

5. You want to use Kong to route requests to two microservices: serviceA at /serviceA and serviceB at /serviceB. Which configuration approach ensures correct routing and avoids path conflicts?

hard

A. Create two routes with the same path ['/service'] for both services

B. Create one route with path ['/'] forwarding to both services

C. Use a single route with no path and rely on backend to differentiate

D. Create two routes with paths ['/serviceA'] and ['/serviceB'], each linked to their respective services

Solution

1. Step 1: Understand routing by path in Kong

   Kong routes requests based on path prefixes to the correct service.

2. Step 2: Avoid path conflicts by using distinct paths

   Separate paths like '/serviceA' and '/serviceB' ensure requests go to the right service without overlap.

3. Final Answer:

   Create two routes with paths ['/serviceA'] and ['/serviceB'], each linked to their respective services -> Option D

4. Quick Check:

   Distinct paths = correct routing [OK]

Hint: Use unique paths per service to avoid conflicts [OK]

Common Mistakes:

• Using same path for multiple services
• Relying on backend to route without gateway paths
• Using root path for all services