JWT token propagation in Microservices - Scalability & System Analysis

Scalability Analysis - JWT token propagation

Growth Table: JWT Token Propagation at Different Scales

Users	Requests per Second (RPS)	Token Size & Overhead	Network Impact	Service Load
100 users	~50 RPS	Small (1-2 KB)	Minimal	Low CPU and memory usage
10,000 users	~5,000 RPS	Small (1-2 KB)	Noticeable increase in bandwidth	Moderate CPU for token verification
1,000,000 users	~500,000 RPS	Small (1-2 KB)	High bandwidth usage, network latency may increase	High CPU usage for token verification; potential bottleneck
100,000,000 users	~50,000,000 RPS	Small (1-2 KB)	Extremely high bandwidth; network saturation risk	Token verification becomes major CPU bottleneck; scaling challenges

First Bottleneck

The first bottleneck in JWT token propagation is the CPU load on microservices caused by verifying tokens on every request. Each service must decode and validate the JWT signature, which is CPU-intensive. As requests grow, this verification step consumes significant CPU resources, slowing down response times.

Network bandwidth can also become a bottleneck at very large scales due to token size added to every request header, but CPU is the primary bottleneck initially.

Scaling Solutions

• Token Verification Caching: Cache verification results for tokens during their validity period to avoid repeated cryptographic checks.
• Offload Verification: Use an API gateway or dedicated authentication service to verify tokens once and pass trusted context downstream.
• Horizontal Scaling: Add more instances of microservices behind load balancers to distribute CPU load.
• Use Lightweight Tokens: Minimize JWT size by including only essential claims to reduce network overhead.
• Asynchronous Processing: For non-critical paths, defer token verification or batch requests.
• Network Optimization: Use HTTP/2 or gRPC to reduce header overhead and improve network efficiency.

Back-of-Envelope Cost Analysis

• At 10,000 users (~5,000 RPS), CPU usage for token verification can reach 30-50% on a single server.
• Each JWT token adds ~1-2 KB per request header, so at 5,000 RPS, bandwidth overhead is ~5-10 MB/s.
• At 1 million users (~500,000 RPS), bandwidth overhead approaches 500-1000 MB/s (4-8 Gbps), requiring high network capacity.
• Storage for tokens is minimal since JWTs are stateless, but caching verification results requires memory proportional to active tokens.

Interview Tip

When discussing JWT token propagation scalability, start by explaining the token verification process and its CPU cost. Then identify the bottleneck (CPU on microservices). Next, propose solutions like caching verification results or offloading verification to a gateway. Discuss network overhead and how to reduce token size. Finally, mention horizontal scaling and network optimizations. Structure your answer by identifying bottlenecks, explaining impact, and proposing targeted fixes.

Self Check

Your database handles 1000 QPS. Traffic grows 10x. What do you do first?

Answer: Since the database is the bottleneck at 1000 QPS, and traffic grows to 10,000 QPS, the first step is to add read replicas and implement connection pooling to distribute load and reduce contention. For JWT token propagation, similarly, if token verification CPU load grows 10x, first add horizontal scaling and caching of verification results before optimizing further.

Key Result

JWT token propagation first breaks at CPU load on microservices due to token verification. Caching and offloading verification, plus horizontal scaling, effectively address this bottleneck.