Connection pooling in HLD - Scalability & System Analysis

Scalability Analysis - Connection pooling

Growth Table: Connection Pooling at Different Scales

Users	Connections Opened	DB Load	Latency Impact	Resource Usage
100 users	~10-50 pooled connections	Low	Minimal	Low CPU & Memory
10,000 users	~200-500 pooled connections	Moderate	Noticeable if no pooling	Moderate CPU & Memory
1,000,000 users	~1000-2000 pooled connections	High	High without pooling	High CPU & Memory, DB saturation risk
100,000,000 users	~5000+ pooled connections (sharded)	Very High	Severe without pooling and sharding	Very High, requires distributed pooling

First Bottleneck

The database server is the first bottleneck. Opening many direct connections for each user request overloads the DB CPU and memory. Without connection pooling, the DB spends too much time managing connections instead of queries.

Scaling Solutions

• Connection Pooling: Reuse a limited number of DB connections for many requests to reduce overhead.
• Horizontal Scaling: Add more app servers each with its own pool to distribute load.
• Read Replicas: Use read-only DB replicas to spread read queries.
• Sharding: Split data across multiple DB instances to reduce single DB load.
• Caching: Cache frequent queries to reduce DB hits.
• Timeouts and Limits: Set max pool size and connection timeouts to avoid resource exhaustion.

Back-of-Envelope Cost Analysis

• 1 server can handle ~1000-5000 concurrent DB connections.
• DB can handle ~5000-10000 queries per second (QPS) depending on hardware.
• Without pooling, each user request may open a new connection causing overhead and latency.
• Pooling reduces connection overhead by reusing connections, improving throughput and reducing CPU usage.
• Bandwidth impact is minimal for connections but important for query result size.

Interview Tip

Start by explaining what connection pooling is and why it matters. Then describe how the DB becomes a bottleneck as users grow. Discuss how pooling reduces connection overhead. Finally, mention complementary scaling techniques like read replicas and sharding.

Self Check

Your database handles 1000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first?

Answer: Implement or increase connection pooling to reuse DB connections efficiently and reduce overhead. Then consider adding read replicas or sharding to distribute load.

Key Result

Connection pooling prevents database overload by reusing a limited number of connections, enabling the system to scale from hundreds to millions of users efficiently.