API authentication (OAuth, JWT, API keys) in HLD - Scalability & System Analysis

Scalability Analysis - API authentication (OAuth, JWT, API keys)

Growth Table: API Authentication Scaling

Users/Requests	Authentication Load	Token Management	Security Concerns	Infrastructure Impact
100 users	Low request rate, simple validation	In-memory token store or stateless JWT	Basic rate limiting	Single auth server sufficient
10,000 users	Moderate request rate, more token checks	Distributed cache for token revocation	Implement throttling, monitor suspicious activity	Load balancer with multiple auth servers
1,000,000 users	High request rate, token validation bottleneck	Stateless JWT preferred, cache public keys, token introspection service	Advanced security: anomaly detection, IP blacklists	Horizontal scaling, CDN for static keys, dedicated auth microservices
100,000,000 users	Very high concurrency, global distribution	Global token key distribution, sharded token revocation, multi-region caches	Zero-trust, continuous authentication, automated threat response	Multi-region clusters, edge authentication, global load balancing

First Bottleneck

The first bottleneck is the authentication server CPU and token validation process. As user requests grow, the server must validate tokens (JWT signature checks, OAuth token introspection, API key lookups) for every request. This CPU-intensive cryptographic operation and database/cache lookups for token revocation or API key validation can overwhelm a single server.

Scaling Solutions

• Horizontal scaling: Add more authentication servers behind a load balancer to distribute token validation load.
• Stateless tokens (JWT): Use JWTs to avoid database lookups on every request by embedding claims and using signature verification.
• Caching: Cache public keys for JWT signature verification and token revocation lists in fast distributed caches like Redis.
• Token expiration: Use short-lived tokens to reduce the need for revocation checks.
• API Gateway: Offload authentication to an API gateway or edge service to reduce backend load.
• Sharding: For API key storage or token revocation data, shard the data across multiple databases or caches.
• CDN for keys: Distribute public keys globally via CDN to speed up JWT verification in multiple regions.

Back-of-Envelope Cost Analysis

• At 1 million users, assuming 1 request per second each, that is 1 million requests/sec.
• Each JWT signature verification takes ~1-5 ms CPU time; 1 million QPS requires thousands of CPU cores or horizontal scaling.
• API key lookups require fast cache or DB queries; Redis can handle ~100K ops/sec per instance, so multiple instances needed.
• Network bandwidth depends on token size (~1 KB); 1 million QPS = ~1 GB/s bandwidth just for tokens.
• Storage for token revocation lists depends on token lifetime and user count; short-lived tokens reduce storage needs.

Interview Tip

Start by explaining the authentication flow simply. Identify the main components: token issuance, validation, and revocation. Discuss how load grows with users and requests. Point out the CPU cost of cryptographic checks and database lookups as bottlenecks. Then propose solutions like stateless JWTs, caching, horizontal scaling, and API gateways. Always justify why each solution fits the bottleneck.

Self Check Question

Your database handles 1000 QPS for API key validation. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add a caching layer (e.g., Redis) in front of the database to serve API key validation requests faster and reduce DB load. If traffic continues to grow, horizontally scale the cache and authentication servers.

Key Result

API authentication first breaks at token validation CPU and database lookups as requests grow; stateless JWTs, caching, and horizontal scaling are key to handle millions of users.

Practice

1. Which of the following is the primary purpose of API authentication?

easy

A. To verify the identity of the client accessing the API

B. To encrypt the data sent between client and server

C. To improve the speed of API responses

D. To log all API requests for analytics

Solution

1. Step 1: Understand the role of authentication

   Authentication is about confirming who is making the request, not about encryption or speed.

2. Step 2: Identify the correct purpose

   Verifying the client's identity ensures only authorized users access the API.

3. Final Answer:

   To verify the identity of the client accessing the API -> Option A

4. Quick Check:

   Authentication = Verify identity [OK]

Hint: Authentication means confirming who you are [OK]

Common Mistakes:

• Confusing authentication with encryption
• Thinking authentication speeds up API
• Mixing authentication with logging

2. Which of the following is the correct format of a JWT token?

easy

A. header.payload.signature

B. username:password

C. api_key=12345

D. token|timestamp|user

Solution

1. Step 1: Recall JWT structure

   A JWT token consists of three parts separated by dots: header, payload, and signature.

2. Step 2: Match the correct format

   Only the format with three dot-separated parts matches JWT.

3. Final Answer:

   header.payload.signature -> Option A

4. Quick Check:

   JWT = header.payload.signature [OK]

Hint: JWT always has three parts separated by dots [OK]

Common Mistakes:

• Confusing JWT with basic auth format
• Using equal signs instead of dots
• Mixing API key format with JWT

3. Given this OAuth flow: A client requests an access token using a refresh token. Which response indicates a successful token refresh?

medium

A. {"refresh_token": "xyz789"}

B. {"error": "invalid_grant"}

C. {"access_token": null}

D. {"access_token": "abc123", "expires_in": 3600}

Solution

1. Step 1: Understand OAuth token refresh response

   A successful refresh returns a new access token and expiry time.

2. Step 2: Identify the success response

   {"access_token": "abc123", "expires_in": 3600} contains a valid access token and expiry, indicating success.

3. Final Answer:

   {"access_token": "abc123", "expires_in": 3600} -> Option D

4. Quick Check:

   Success = new access token returned [OK]

Hint: Success returns new access token, not error or null [OK]

Common Mistakes:

• Confusing error response with success
• Expecting refresh token in response
• Assuming null access token means success

4. A developer uses an API key in the URL query string for authentication but notices security warnings. What is the main issue?

medium

A. API keys cannot be used for authentication

B. API keys must be encrypted with RSA

C. API keys in URLs can be logged or leaked easily

D. API keys expire too quickly

Solution

1. Step 1: Analyze API key usage in URLs

   URLs are often logged by servers, browsers, and proxies, risking key exposure.

2. Step 2: Identify the security risk

   Embedding API keys in URLs can leak them unintentionally, causing security warnings.

3. Final Answer:

   API keys in URLs can be logged or leaked easily -> Option C

4. Quick Check:

   API key in URL = risk of leakage [OK]

Hint: Never put API keys in URLs to avoid leaks [OK]

Common Mistakes:

• Thinking API keys must be encrypted always
• Believing API keys can't authenticate
• Assuming keys expire too fast

5. You design an API that uses JWT for authentication. To improve security, which combination is best practice?

hard

A. Store JWTs in localStorage and never expire them

B. Use short-lived JWTs with HTTPS and validate signature on each request

C. Send JWTs in URL query parameters for easy access

D. Use JWTs without signature to reduce overhead

Solution

1. Step 1: Identify secure JWT handling practices

   Short-lived tokens reduce risk if stolen; HTTPS protects data in transit; signature validation ensures token integrity.

2. Step 2: Evaluate options for security

   Use short-lived JWTs with HTTPS and validate signature on each request follows all best practices; others expose security risks like token theft or tampering.

3. Final Answer:

   Use short-lived JWTs with HTTPS and validate signature on each request -> Option B

4. Quick Check:

   Secure JWT = short life + HTTPS + signature check [OK]

Hint: Short life + HTTPS + signature check = secure JWT [OK]

Common Mistakes:

• Storing JWTs insecurely in localStorage
• Sending tokens in URLs risking leaks
• Skipping signature validation