Secrets management (Vault, AWS Secrets Manager) in Microservices - Scalability & System Analysis

Scalability Analysis - Secrets management (Vault, AWS Secrets Manager)

Growth Table: Secrets Management Scaling

Scale	Number of Microservices	Secrets Stored	Request Rate (QPS)	Key Changes
100 users	10-20	100-500	50-200	Single Vault/AWS Secrets Manager instance; low latency; simple access policies
10,000 users	100-200	5,000-10,000	1,000-5,000	Introduce caching at microservice side; enable read replicas; fine-grained access control
1,000,000 users	1,000+	100,000+	50,000-100,000	Use distributed Vault clusters or multi-region AWS Secrets Manager; heavy caching; rate limiting; secrets rotation automation
100,000,000 users	10,000+	1,000,000+	500,000+	Global multi-region deployment; sharding secrets by service or region; advanced monitoring; strict quota enforcement

First Bottleneck

The first bottleneck is the secrets storage backend (Vault or AWS Secrets Manager). At moderate scale, the backend can become overwhelmed by high QPS of secret read requests from many microservices, causing increased latency and throttling.

Scaling Solutions

• Caching: Implement local caching of secrets in microservices with TTL to reduce backend calls.
• Read Replicas: Use Vault clusters or AWS Secrets Manager replicas to distribute read load.
• Horizontal Scaling: Deploy multiple Vault nodes behind a load balancer or use multi-region AWS Secrets Manager.
• Sharding: Partition secrets by service or region to reduce contention.
• Rate Limiting: Enforce request quotas to prevent overload.
• Automation: Automate secret rotation and renewal to avoid stale secrets and reduce manual overhead.

Back-of-Envelope Cost Analysis

• At 10,000 QPS, assuming each secret read is ~1KB, bandwidth = 10,000 KB/s (~10 MB/s).
• Storage: For 100,000 secrets averaging 1KB each, total storage ~100 MB (small, but grows with metadata and versions).
• CPU/Memory: Vault nodes need enough CPU to handle encryption/decryption and network I/O; AWS Secrets Manager is managed but costs scale with requests.
• Network: Ensure network capacity to handle peak QPS without latency spikes.

Interview Tip

Start by identifying the main components: secrets storage, microservices, and access patterns. Discuss bottlenecks focusing on request rates and latency. Propose caching and replication early. Highlight security concerns like access control and rotation. Structure your answer by scale and how each solution addresses specific bottlenecks.

Self Check Question

Your database handles 1000 QPS for secret reads. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Implement caching at the microservice level to reduce direct reads from the secrets backend, and add read replicas or scale Vault nodes horizontally to distribute load.

Key Result

Secrets storage backend becomes the first bottleneck as request rates grow; caching and replication are key to scaling.