API key management in Microservices - Scalability & System Analysis

Scalability Analysis - API key management

Growth Table: API Key Management at Different Scales

Users / Requests	100 Users	10K Users	1M Users	100M Users
API Requests per Second (RPS)	~50 RPS	~5,000 RPS	~500,000 RPS	~50,000,000 RPS
API Key Storage Size	~100 keys (KBs)	~10K keys (MBs)	~1M keys (GBs)	~100M keys (100s GBs)
Authentication Latency	<1 ms	~1-5 ms	~5-20 ms	20+ ms without optimization
Rate Limiting Complexity	Simple in-memory counters	Distributed counters needed	Sharded counters with caching	Global distributed rate limiting system
Security Measures	Basic encryption and logging	Enhanced encryption, audit logs	Advanced monitoring, anomaly detection	AI-based threat detection, automated key revocation

First Bottleneck

The first bottleneck is the database that stores API keys and usage data. At low scale, a single database instance can handle key lookups and updates. As traffic grows to thousands of requests per second, the database faces high read/write loads for authentication and rate limiting.

Without caching, each API request triggers a database read to validate the key and update usage counters, causing latency and throughput issues.

Scaling Solutions

• Caching: Use an in-memory cache (e.g., Redis) to store API key validation results and rate limit counters to reduce database load.
• Read Replicas: Add read replicas for the database to distribute read queries.
• Sharding: Partition API keys by user ID or key prefix to distribute data and load across multiple database instances.
• Horizontal Scaling: Add more authentication servers behind a load balancer to handle increased request volume.
• Rate Limiting: Implement distributed rate limiting using Redis or specialized services to handle counters efficiently.
• Security: Use encryption for stored keys, rotate keys regularly, and monitor usage patterns for anomalies.

Back-of-Envelope Cost Analysis

• At 10K users generating ~5,000 RPS, assuming each request requires 2 Redis operations (check + increment), Redis handles ~10,000 ops/sec, well within a single instance capacity.
• Database writes for usage logs can be batched or asynchronously processed to reduce load.
• Storage for 1M API keys with metadata (~1 KB per key) requires ~1 GB of storage.
• Network bandwidth depends on request size; assuming 1 KB per request, 500,000 RPS equals ~500 MB/s, requiring multiple servers and network interfaces.

Interview Tip

When discussing API key management scalability, start by explaining the key components: storage, authentication, rate limiting, and security. Then describe how load increases affect each component. Identify the database as the first bottleneck and propose caching and sharding as solutions. Discuss trade-offs like consistency vs. latency. Finally, mention monitoring and security as ongoing concerns.

Self Check Question

Question: Your database handles 1000 QPS for API key validation. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: Add a caching layer (e.g., Redis) to store API key validation results and rate limit counters. This reduces direct database reads and writes, lowering latency and increasing throughput before scaling the database.

Key Result

The database storing API keys and usage data is the first bottleneck as traffic grows; adding caching and sharding are key steps to scale API key management efficiently.