CDN caching for static content in HLD - System Design Exercise

Design: CDN caching for static content

Design focuses on caching static content using CDN edge servers, cache invalidation, and request routing. Does not cover dynamic content caching or origin server design.

Functional Requirements

FR1: Serve static content (images, CSS, JavaScript) to users globally with low latency

FR2: Cache static content at edge locations close to users

FR3: Support cache invalidation when content updates

FR4: Handle at least 1 million requests per second globally

FR5: Ensure high availability with 99.9% uptime

FR6: Support HTTPS for secure content delivery

Non-Functional Requirements

NFR1: Latency for content delivery should be under 100ms for 95% of requests

NFR2: Cache hit ratio should be above 90% to reduce origin load

NFR3: System must handle traffic spikes up to 5x normal load

NFR4: Content updates should propagate to edge caches within 5 minutes

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Origin server for static content

CDN edge servers distributed globally

DNS-based request routing

Cache storage and eviction policies

Cache invalidation mechanism

TLS termination for HTTPS

Design Patterns

Cache-aside pattern

Time-to-live (TTL) based caching

Push vs pull cache invalidation

Geo DNS routing

Load balancing at origin

Reference Architecture

          +-------------------+
          |   User Devices    |
          +---------+---------+
                    |
                    v
          +---------+---------+          +----------------+
          |    DNS Resolver   |--------->| Geo DNS Routing |
          +---------+---------+          +--------+-------+
                    |                             |
                    v                             v
       +------------+------------+     +----------+----------+
       |   CDN Edge Server 1     | ... |  CDN Edge Server N   |
       | (Cache + TLS Termination)|     | (Cache + TLS Termination)|
       +------------+------------+     +----------+----------+
                    |                             |
                    +-------------+---------------+
                                  |
                                  v
                         +--------+--------+
                         |   Origin Server  |
                         | (Static Content) |
                         +------------------+

Components

Origin Server

Any HTTP server (e.g., Nginx, Apache)

Stores original static content and serves cache misses

CDN Edge Servers

Distributed cache servers at global edge locations

Cache static content close to users to reduce latency

Geo DNS Routing

DNS service with geographic routing (e.g., Route53 Geo DNS)

Direct user requests to nearest CDN edge server

Cache Storage

In-memory or SSD cache on edge servers

Store cached static content with TTL and eviction policies

Cache Invalidation System

API or push mechanism to purge or update cache entries

Ensure updated content is refreshed on edge caches

TLS Termination

Edge server SSL/TLS certificates and termination

Serve HTTPS content securely from edge

Request Flow

1. User requests static content URL.

2. DNS resolver queries Geo DNS routing to find nearest CDN edge server.

3. User request is routed to closest CDN edge server.

4. Edge server checks cache for requested content.

5. If cache hit, edge server returns content immediately.

6. If cache miss, edge server requests content from origin server.

7. Origin server responds with static content.

8. Edge server caches content with TTL and returns to user.

9. When content updates, origin triggers cache invalidation API.

10. Edge servers purge or update cached content accordingly.

Database Schema

Not applicable as system primarily uses cache storage and origin file storage.

Scaling Discussion

Bottlenecks

Origin server overload due to cache misses during traffic spikes

DNS routing delays or failures affecting request routing

Cache storage limits on edge servers causing frequent evictions

Cache invalidation delays causing stale content delivery

TLS termination CPU overhead on edge servers

Solutions

Use origin load balancing and auto-scaling to handle cache miss spikes

Use highly available and globally distributed DNS services

Increase cache storage capacity and optimize eviction policies

Implement near real-time push-based cache invalidation

Use hardware acceleration or dedicated TLS termination services

Interview Tips

Time: 10 minutes for requirements and clarifications, 15 minutes for architecture and data flow, 10 minutes for scaling and trade-offs, 10 minutes for Q&A

Clarify content types and update frequency

Explain CDN edge caching benefits and cache hit impact

Describe DNS-based geo routing for latency optimization

Discuss cache invalidation strategies and TTL trade-offs

Address scaling bottlenecks and mitigation techniques

Highlight security with HTTPS and TLS termination