First design walkthrough (URL shortener) in HLD - System Design Exercise

Design: URL Shortener

Design covers URL shortening, redirection, and basic analytics. User authentication, advanced analytics, and monetization are out of scope.

Functional Requirements

FR1: Generate a short alias for a given long URL

FR2: Redirect users from the short URL to the original long URL

FR3: Handle at least 100 million URLs

FR4: Support 10,000 requests per second for redirection

FR5: Ensure short URLs are unique and not easily guessable

FR6: Provide basic analytics: number of clicks per short URL

FR7: Allow users to customize the short URL alias (optional)

Non-Functional Requirements

NFR1: Latency for redirection should be under 100ms (p99)

NFR2: Availability target of 99.9% uptime

NFR3: Storage must be scalable to handle growth

NFR4: System should prevent collisions in short URL generation

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

API Gateway or Load Balancer

URL Shortening Service

Database for storing URL mappings

Cache layer for fast redirection

Analytics Service

Unique ID generator or hashing mechanism

Design Patterns

Hashing and Base62 encoding for short URL generation

Cache-Aside pattern for caching URL mappings

Database sharding for scalability

Rate limiting to prevent abuse

Asynchronous processing for analytics

Reference Architecture

API Gateway / Load Balancer

↓

URL Shortening Service

↓

Cache

↓

Analytics Service

Components

API Gateway / Load Balancer

Nginx or AWS ALB

Distribute incoming requests and provide a single entry point

URL Shortening Service

Stateless microservice (e.g., Node.js, Python Flask)

Generate short URLs, handle redirection, and validate requests

Database

Relational DB like PostgreSQL or NoSQL like Cassandra

Store mappings between short URLs and original URLs

Cache

Redis or Memcached

Store frequently accessed URL mappings for fast redirection

Analytics Service

Background worker with message queue (e.g., Kafka, RabbitMQ)

Collect and process click data asynchronously

Unique ID Generator

Base62 encoding of auto-increment ID or hash function

Create unique, short, and collision-resistant aliases

Request Flow

1. User sends a request to shorten a URL via API Gateway.

2. URL Shortening Service validates the URL and checks for custom alias if provided.

3. If no custom alias, service generates a unique ID and encodes it to create short URL.

4. Mapping of short URL to original URL is stored in the database.

5. Short URL is returned to the user.

6. When a user accesses the short URL, request goes to API Gateway then URL Shortening Service.

7. Service checks cache for the original URL; if not found, queries the database.

8. Redirects user to the original URL.

9. Click event is sent asynchronously to Analytics Service for processing.

Database Schema

Entities: - URLMapping: id (PK), short_url (unique), original_url, creation_date, custom_alias (nullable), click_count - Analytics: short_url (FK), timestamp, user_agent, ip_address Relationships: - URLMapping has many Analytics records - short_url is unique and indexed for fast lookup

Scaling Discussion

Bottlenecks

Database write and read throughput limits with growing URL mappings

Cache size and eviction policies under heavy read load

Unique ID generation collisions or bottlenecks

Analytics processing delays with high click volume

API Gateway handling large concurrent requests

Solutions

Shard database by URL hash or user to distribute load

Use distributed cache clusters with consistent hashing

Use distributed ID generators like Snowflake or UUIDs with collision checks

Process analytics asynchronously with scalable message queues and workers

Auto-scale API Gateway and use CDN for static content

Interview Tips

Time: Spend 10 minutes clarifying requirements and constraints, 20 minutes designing components and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Clarify assumptions and requirements before starting design

Explain choice of unique ID generation and collision avoidance

Discuss caching strategy to reduce latency

Highlight asynchronous analytics processing for scalability

Address how to handle scaling bottlenecks realistically

Mention trade-offs between consistency and availability