First design walkthrough (URL shortener) in HLD - Scalability & System Analysis

Scalability Analysis - First design walkthrough (URL shortener)

Growth Table: URL Shortener Scaling

Scale	Users	Requests per Second (QPS)	Data Stored	Key Changes
Small	100 users	~10 QPS	~10K URLs (~1MB)	Single server, single DB instance, simple hashing
Medium	10,000 users	~1,000 QPS	~1M URLs (~100MB)	DB read replicas, caching hot URLs, load balancer
Large	1,000,000 users	~100,000 QPS	~100M URLs (~10GB)	Sharded DB, distributed cache, CDN for redirects
Very Large	100,000,000 users	~10M QPS	~10B URLs (~1TB+)	Multi-region sharding, global CDN, asynchronous writes, tiered storage

First Bottleneck

At small to medium scale, the database is the first bottleneck. It handles all URL mappings and redirect lookups. As traffic grows, the DB CPU and I/O get overwhelmed by read and write queries.

At large scale, the application servers can become CPU-bound due to encoding/decoding URLs and handling many concurrent connections.

At very large scale, network bandwidth and data partitioning become critical challenges.

Scaling Solutions

• Database: Add read replicas to distribute read load. Use connection pooling. Implement sharding by URL hash ranges to split data across servers.
• Caching: Use in-memory caches (like Redis) for popular URLs to reduce DB hits.
• Application Servers: Horizontally scale by adding more servers behind a load balancer.
• CDN: Use a Content Delivery Network to cache redirect responses globally, reducing latency and load.
• Storage: Archive old or rarely used URLs to cheaper storage tiers.
• Asynchronous Processing: Use queues for write-heavy operations to smooth spikes.

Back-of-Envelope Cost Analysis

• At 1,000 QPS, DB needs to handle ~86 million requests per day.
• Each URL record ~100 bytes (short URL, original URL, metadata).
• 1 million URLs ~100 MB storage; 100 million URLs ~10 GB.
• Redirect responses are small (~100 bytes), so bandwidth at 100,000 QPS ~10 MB/s (800 Mbps).
• Network bandwidth and storage costs grow with scale; caching reduces DB and bandwidth load.

Interview Tip

Start by clarifying requirements: expected traffic, URL length, expiration, analytics.

Discuss data model and how to generate unique short URLs.

Explain bottlenecks at different scales and propose solutions step-by-step.

Use real numbers to justify design choices.

Show awareness of trade-offs like consistency vs availability.

Self Check Question

Your database handles 1,000 QPS. Traffic grows 10x to 10,000 QPS. What do you do first and why?

Answer: Add read replicas and implement caching to reduce DB load, because the DB is the first bottleneck at this scale.

Key Result

The database is the first bottleneck as traffic grows; scaling requires read replicas, caching, sharding, and horizontal scaling of app servers with CDN support for redirects.