API authentication (OAuth, JWT, API keys) in HLD - Scalability & System Analysis

Scalability Analysis - API authentication (OAuth, JWT, API keys)

Growth Table: API Authentication Scaling

Users/Requests	Authentication Load	Token Management	Security Concerns	Infrastructure Impact
100 users	Low request rate, simple validation	In-memory token store or stateless JWT	Basic rate limiting	Single auth server sufficient
10,000 users	Moderate request rate, more token checks	Distributed cache for token revocation	Implement throttling, monitor suspicious activity	Load balancer with multiple auth servers
1,000,000 users	High request rate, token validation bottleneck	Stateless JWT preferred, cache public keys, token introspection service	Advanced security: anomaly detection, IP blacklists	Horizontal scaling, CDN for static keys, dedicated auth microservices
100,000,000 users	Very high concurrency, global distribution	Global token key distribution, sharded token revocation, multi-region caches	Zero-trust, continuous authentication, automated threat response	Multi-region clusters, edge authentication, global load balancing

First Bottleneck

The first bottleneck is the authentication server CPU and token validation process. As user requests grow, the server must validate tokens (JWT signature checks, OAuth token introspection, API key lookups) for every request. This CPU-intensive cryptographic operation and database/cache lookups for token revocation or API key validation can overwhelm a single server.

Scaling Solutions

• Horizontal scaling: Add more authentication servers behind a load balancer to distribute token validation load.
• Stateless tokens (JWT): Use JWTs to avoid database lookups on every request by embedding claims and using signature verification.
• Caching: Cache public keys for JWT signature verification and token revocation lists in fast distributed caches like Redis.
• Token expiration: Use short-lived tokens to reduce the need for revocation checks.
• API Gateway: Offload authentication to an API gateway or edge service to reduce backend load.
• Sharding: For API key storage or token revocation data, shard the data across multiple databases or caches.
• CDN for keys: Distribute public keys globally via CDN to speed up JWT verification in multiple regions.

Back-of-Envelope Cost Analysis

• At 1 million users, assuming 1 request per second each, that is 1 million requests/sec.
• Each JWT signature verification takes ~1-5 ms CPU time; 1 million QPS requires thousands of CPU cores or horizontal scaling.
• API key lookups require fast cache or DB queries; Redis can handle ~100K ops/sec per instance, so multiple instances needed.
• Network bandwidth depends on token size (~1 KB); 1 million QPS = ~1 GB/s bandwidth just for tokens.
• Storage for token revocation lists depends on token lifetime and user count; short-lived tokens reduce storage needs.

Interview Tip

Start by explaining the authentication flow simply. Identify the main components: token issuance, validation, and revocation. Discuss how load grows with users and requests. Point out the CPU cost of cryptographic checks and database lookups as bottlenecks. Then propose solutions like stateless JWTs, caching, horizontal scaling, and API gateways. Always justify why each solution fits the bottleneck.

Self Check Question

Your database handles 1000 QPS for API key validation. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Add a caching layer (e.g., Redis) in front of the database to serve API key validation requests faster and reduce DB load. If traffic continues to grow, horizontally scale the cache and authentication servers.

Key Result

API authentication first breaks at token validation CPU and database lookups as requests grow; stateless JWTs, caching, and horizontal scaling are key to handle millions of users.