Config server pattern in Microservices - Scalability & System Analysis

Scalability Analysis - Config server pattern

Growth Table: Config Server Pattern Scaling

Users / Services	100 Users / 10 Services	10K Users / 100 Services	1M Users / 1000 Services	100M Users / 10,000 Services
Config Requests per Second	~50-100	~5,000	~50,000	~500,000
Config Server Instances	1-2	3-5 (load balanced)	10-20 (clustered)	50+ (sharded clusters)
Config Data Size	Small (KBs)	Medium (MBs)	Large (100s MBs)	Very Large (GBs)
Cache Usage	Basic in-memory cache	Distributed cache (Redis)	Multi-level cache + CDN	Global CDN + edge caches
Network Bandwidth	Low	Moderate	High	Very High

First Bottleneck

The first bottleneck is the config server itself. As the number of microservices and users grow, the config server faces high read traffic for configuration data. Without caching, the server CPU and memory get overwhelmed. Also, the underlying storage (e.g., Git repo or database) can become slow under heavy load.

Scaling Solutions

• Horizontal scaling: Add more config server instances behind a load balancer to distribute requests.
• Caching: Use in-memory caches on config servers and distributed caches like Redis to reduce backend hits.
• Sharding: Partition config data by service groups or environments to reduce single server load.
• CDN / Edge caching: For static config files, use CDN to serve configs closer to services.
• Asynchronous updates: Push config changes to services instead of frequent polling to reduce request volume.
• Optimize storage: Use fast storage like SSDs or in-memory databases for config data.

Back-of-Envelope Cost Analysis

Assuming 1000 microservices each polling config every 30 seconds:

• Requests per second = 1000 services * (1/30) = ~33 QPS
• At 1M users with 1000 services, QPS can reach 50,000+ due to retries and bursts.
• Config data size per request ~10 KB -> Bandwidth = 50,000 * 10 KB = ~500 MB/s (~4 Gbps)
• Storage needs depend on config versions; typically a few GBs but grows with history.
• CPU and memory on config servers must handle peak QPS with caching to avoid overload.

Interview Tip

Start by explaining the config server role and typical traffic patterns. Identify the bottleneck as the config server under read load. Discuss caching and horizontal scaling first. Then mention sharding and CDN for large scale. Always connect solutions to the bottleneck you identified. Use simple analogies like a library lending books (config data) to many readers (services).

Self Check

Question: Your config server handles 1000 QPS. Traffic grows 10x. What do you do first and why?

Answer: Add caching to reduce backend load and horizontally scale config server instances behind a load balancer. This reduces CPU/memory bottleneck and spreads traffic.

Key Result

The config server pattern scales well initially but the config server becomes the first bottleneck under high read traffic. Caching and horizontal scaling are key first steps to handle growth.