CDN caching for static content in HLD - Scalability & System Analysis

Scalability Analysis - CDN caching for static content

Growth Table: CDN Caching for Static Content

Users	Requests per Second	Cache Hit Ratio	Origin Server Load	Bandwidth Usage	Latency
100 users	~10-50 RPS	~80%	Low	Low	Low
10,000 users	~1,000-5,000 RPS	~85-90%	Moderate	Moderate	Low
1,000,000 users	~100,000-500,000 RPS	~90-95%	High without CDN scaling	High	Low to Moderate
100,000,000 users	~10M-50M RPS	~95-99%	Origin offloaded heavily	Very High	Low (global edge caching)

First Bottleneck

The origin server becomes the first bottleneck as user requests grow. Without CDN caching, the origin must serve all static content, causing high CPU, bandwidth, and latency issues. Network bandwidth at the origin and server capacity limit how many requests it can handle.

Scaling Solutions

• Use CDN edge caching: Cache static content at multiple global edge locations to serve users closer to them, reducing origin load and latency.
• Cache control headers: Set proper cache expiration and validation headers to maximize cache hit ratio and freshness.
• Cache invalidation: Implement efficient cache purging or versioning to update content without overloading origin.
• Origin scaling: Scale origin servers vertically or horizontally as a fallback when cache misses occur.
• Compression and optimization: Compress static files and optimize assets to reduce bandwidth usage.
• Load balancing: Distribute origin requests across multiple servers or regions.

Back-of-Envelope Cost Analysis

• At 1M users with 100K RPS, assuming 90% cache hit, origin handles 10K RPS.
• Origin bandwidth: For 1MB average static file, 10K RPS = ~10 GB/s (~80 Gbps), which requires high network capacity.
• CDN edges handle 90K RPS, distributing bandwidth globally, reducing origin costs.
• Storage: Static content storage is usually small (GBs to TBs), but must be replicated across CDN nodes.
• Cost savings come from reduced origin bandwidth and server load.

Interview Tip

Start by explaining the role of CDN caching in reducing origin load and latency. Discuss cache hit ratio impact and how cache control headers affect freshness. Then, identify bottlenecks at origin servers and network bandwidth. Finally, propose scaling solutions like edge caching, cache invalidation, and origin scaling. Use numbers to justify your points.

Self Check

Your origin server handles 1000 RPS for static content. Traffic grows 10x. What do you do first?

Answer: Deploy CDN edge caching for static content to offload the origin server and reduce load before scaling origin servers vertically or horizontally.

Key Result

CDN caching shifts static content delivery from origin servers to global edge locations, drastically reducing origin load and latency as user traffic grows from thousands to millions.