Bird
Raised Fist0

Design a URL Shortener - Encoding, Redirect, Analytics, Custom Alias

HLDmediumInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Design a URL Shortener - Encoding, Redirect, Analytics, Custom Alias
45 min · senior
senior

You are designing a URL shortener service for a startup with 50 million daily active users. The system must generate short URLs that redirect to original URLs with P99 latency under 200ms globally. It must support custom aliases, encrypt stored URLs for privacy compliance, and provide analytics on usage (click counts, geolocation, device info). The system should handle 10 million new short URLs per day and 500 million redirects per day.

Constraints
  • All original URLs must be encrypted at rest using AES-256 to comply with privacy regulations.
  • Redirect latency P99 must be under 200ms globally, including DNS lookup and network overhead.
  • Support custom aliases with collision detection and prevention.
  • Analytics data must be eventually consistent and support querying click counts by URL and geolocation.
  • The system must handle 10 million new short URLs and 500 million redirects daily.
  • Short URLs must be case-sensitive and use a base62 encoding scheme.
📋
Guided Steps
1/5
Step 1
Choosing the Storage for URL Mapping
Which storage system will you use to store the mapping from short URL keys to encrypted original URLs, and why?
💡 Hint
Consider write throughput for 10M new URLs/day and low-latency reads for 500M redirects/day.
Key Decision
Use Amazon DynamoDB with on-demand capacity and encryption at rest enabled - it supports single-digit millisecond reads/writes at scale, built-in encryption, and seamless scaling for 10M writes/day and 500M reads/day.
Interviewer Expectation
Mid-level: Names DynamoDB or Cassandra. Senior: Explains DynamoDB’s on-demand scaling, encryption at rest, and low-latency single-key lookups versus Cassandra’s operational complexity and eventual consistency trade-offs.
⚠️Common Wrong Answer
Using a relational database like MySQL for URL mapping - it cannot handle 500M reads/day with <200ms P99 latency and causes hotspots on primary keys.
Step 2
Encoding and Collision Handling for Short URLs
How will you generate short URL keys and handle collisions, especially for custom aliases?
💡 Hint
Think about base62 encoding, uniqueness guarantees, and atomic checks for custom aliases.
Key Decision
Generate short keys using a base62 encoding of a 64-bit unique ID from a Snowflake ID generator; for custom aliases, perform atomic conditional writes in DynamoDB to prevent collisions.
Interviewer Expectation
Mid-level: Describes base62 encoding and uniqueness. Senior: Explains Snowflake ID’s distributed uniqueness, why hashing original URLs is insufficient due to collisions, and how DynamoDB conditional writes prevent race conditions on custom aliases.
⚠️Common Wrong Answer
Using MD5 hash truncated to 6 characters for keys - leads to high collision probability and no atomic collision prevention for custom aliases.
Step 3
Redirect Path and Latency Optimization
How will the redirect endpoint resolve short URLs to original URLs with P99 latency under 200ms globally?
💡 Hint
Consider caching layers, CDN usage, and decryption overhead.
Key Decision
Use Amazon CloudFront CDN in front of a stateless redirect service; cache decrypted original URLs in Redis with TTL to reduce DynamoDB reads; decrypt URLs on cache miss using AES-256 with AWS KMS for key management.
Interviewer Expectation
Mid-level: Mentions caching and CDN. Senior: Explains cache TTL tuning to balance freshness and load, CloudFront edge caching to reduce latency globally, and AWS KMS integration for secure, low-latency decryption.
⚠️Common Wrong Answer
Redirect service queries DynamoDB and decrypts every request without caching - causes high latency and throttling under 500M daily redirects.
Step 4
Analytics Data Collection and Querying
How will you collect and serve analytics data such as click counts and geolocation for short URLs?
💡 Hint
Think about event streaming, aggregation, and eventual consistency.
Key Decision
Use Amazon Kinesis Data Streams to ingest click events from redirect service; process events with AWS Lambda to aggregate counts by URL and geolocation; store aggregates in Amazon DynamoDB for fast querying.
Interviewer Expectation
Mid-level: Names event streaming and aggregation. Senior: Explains Kinesis’s high throughput for 500M events/day, Lambda’s serverless scaling, and DynamoDB’s fast key-value lookups for analytics queries with eventual consistency.
⚠️Common Wrong Answer
Writing every click event synchronously to a relational database - causes write bottlenecks and high latency.
Step 5
API Design for Core Endpoints
Design the API endpoints for creating short URLs, redirecting, and fetching analytics. What are the request and response formats?
💡 Hint
Consider RESTful design, idempotency, and error handling.
Key Decision
POST /shorten with JSON body {original_url, optional custom_alias} returns {short_url}; GET /{short_key} performs redirect with 302 status; GET /analytics/{short_key} returns JSON with click counts and geolocation breakdown.
Interviewer Expectation
Mid-level: Defines basic endpoints and parameters. Senior: Details idempotency for POST, validation for custom aliases, HTTP status codes for errors, and pagination for analytics data.
⚠️Common Wrong Answer
Using GET for URL creation or returning 200 OK on redirect instead of 302 - breaks REST semantics and client expectations.
🔌
API Design Challenge
Sketch the design of 3 core API endpoints for the URL shortener: creating a short URL (with optional custom alias), redirecting a short URL to the original URL, and fetching analytics for a short URL.
Hints
  • How does the client submit the original URL and optional custom alias?
  • How does the redirect endpoint handle cache misses and encryption?
  • How does the analytics endpoint aggregate and serve click data efficiently?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Use of a scalable NoSQL store like DynamoDB for URL mapping with encryption
  • Base62 encoding of unique IDs for short URL keys
  • Atomic collision prevention for custom aliases
  • Caching and CDN usage to meet redirect latency SLAs
  • Event streaming for analytics data ingestion
Good to Have - 30%
  • Use of AWS KMS for encryption key management
  • Redis caching layer for decrypted URLs
  • Idempotent and RESTful API design
  • Detailed latency and throughput estimations
Bonus - 10%
  • Discussion of multi-region replication for high availability
  • Consideration of abuse prevention (rate limiting, spam detection)
Anti-patterns to Avoid
✅
Model Answer
▼ Reveal

This design uses Amazon DynamoDB with on-demand capacity and encryption at rest to store mappings from short keys to encrypted original URLs, supporting 10M writes and 500M reads daily with low latency. Short keys are generated by base62 encoding 64-bit unique IDs from a Snowflake ID generator, ensuring uniqueness and scalability. Custom aliases are supported with atomic conditional writes in DynamoDB to prevent collisions. Redirects are served via a stateless service behind Amazon CloudFront CDN, with Redis caching decrypted URLs to meet P99 latency under 200ms globally. Original URLs are encrypted using AES-256 with AWS KMS for key management to ensure privacy compliance. Analytics data is ingested via Amazon Kinesis Data Streams, processed by AWS Lambda for aggregation, and stored in DynamoDB for fast querying with eventual consistency. The API design includes RESTful endpoints for URL creation (POST /shorten), redirect (GET /{short_key}), and analytics retrieval (GET /analytics/{short_key}), with idempotency, validation, and proper HTTP status codes.

Key Decisions
  • Use DynamoDB for scalable, low-latency storage with encryption at rest
  • Generate short keys by base62 encoding Snowflake-generated unique IDs
  • Prevent custom alias collisions with DynamoDB conditional writes
  • Use CloudFront CDN and Redis caching to meet redirect latency SLAs
  • Encrypt original URLs with AES-256 using AWS KMS for secure key management
  • Ingest analytics events via Kinesis and aggregate with Lambda, storing results in DynamoDB
  • Design RESTful APIs with POST for creation, GET for redirect and analytics, ensuring idempotency and proper error handling
  • Amazon DynamoDB with on-demand capacity and encryption at rest for URL mapping - supports 10M writes/day and 500M reads/day with <10ms latency
  • Snowflake ID generator for unique 64-bit IDs encoded in base62 - ensures distributed uniqueness and short keys
  • DynamoDB conditional writes for atomic custom alias collision prevention - avoids race conditions
  • Amazon CloudFront CDN for global edge caching of redirect responses - reduces latency under 200ms P99
  • Redis cache with TTL for decrypted original URLs - reduces DynamoDB read load and decryption overhead
  • AES-256 encryption of original URLs with AWS KMS for secure key management - meets privacy compliance
  • Amazon Kinesis Data Streams and AWS Lambda for scalable analytics event ingestion and aggregation
  • RESTful API design with POST /shorten, GET /{short_key} redirect, and GET /analytics/{short_key} endpoints

Practice

(1/5)
1. In the request flow, at step 5 the Intent Classifier component receives a user query. What happens if the confidence score is below the threshold at this step?
easy
A. The query is logged and a fallback notification is triggered asynchronously
B. The query is immediately sent to the human agent without further processing
C. The query is discarded and the user is asked to rephrase
D. The query is forwarded to the Fallback Handler component for alternative resolution

Solution

  1. Step 1: Trace Intent Classifier decision at Step 5

    When confidence is low, the system logs the query and triggers fallback notification asynchronously to avoid blocking.

  2. Step 2: Confirm fallback handling behavior

    Immediate human routing (A) blocks scalability. Forwarding directly (B) is plausible but synchronous. Discarding (C) harms UX. Logging and async notification (D) balances reliability and responsiveness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Low confidence -> async fallback notification and logging [check]
Hint: Low confidence triggers async fallback notification [check]
Common Mistakes:
2. For global video delivery, which architecture better handles sudden traffic spikes in a specific region with minimal latency increase?
medium
A. Hierarchical CDN with regional edge caches and origin fallback
B. Centralized origin server cluster with global load balancer
C. Peer-to-peer streaming where clients share video chunks directly
D. Multiple independent origin servers in each region without caching

Solution

  1. Step 1: Evaluate architectures for traffic spike handling

    Centralized origin (Centralized origin server cluster with global load balancer) risks overload and latency spikes. Peer-to-peer (Peer-to-peer streaming where clients share video chunks directly) depends on client availability and is complex for global scale. Independent origins without caching (Multiple independent origin servers in each region without caching) lack scalability and increase origin load.

  2. Step 2: Assess hierarchical CDN benefits

    Regional edge caches (Hierarchical CDN with regional edge caches and origin fallback) absorb traffic spikes locally, reducing origin load and latency, providing scalable and resilient delivery.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Hierarchical CDN absorbs regional spikes with low latency [check]
Hint: Regional caches absorb spikes, origin fallback reduces load [check]
Common Mistakes:
3. A startup expects rapid feature growth but limited initial traffic. They must choose between a monolith and microservices. Which architecture is best for minimizing early development overhead while allowing future scaling?
medium
A. Start with microservices to enforce strict service boundaries early
B. Start with a modular monolith and refactor to microservices when needed
C. Use a serverless architecture to avoid managing services
D. Build multiple small monoliths for each feature to isolate code

Solution

  1. Step 1: Evaluate development overhead vs scaling needs

    Serverless reduces initial infrastructure management overhead by outsourcing scaling and deployment.

  2. Step 2: Consider future scaling and limitations

    Serverless can limit control and cause cold start latency, but suits rapid early development with limited traffic.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Serverless minimizes early overhead but may limit control later [check]
Hint: Serverless reduces early overhead but limits control [check]
Common Mistakes:
4. At 10x scale, which component in a consistent hashing database sharding system is most likely to fail first and why?
hard
A. The central shard metadata service, due to increased coordination overhead
B. The individual database shards, because storage capacity limits are reached first
C. The hash ring management service, because updating and propagating ring changes becomes a bottleneck
D. The client-side routing cache, as it cannot handle the increased number of shards efficiently

Solution

  1. Step 1: Identify scaling bottlenecks

    Central metadata service (The central shard metadata service, due to increased coordination overhead) may be distributed or avoided; shards (The individual database shards, because storage capacity limits are reached first) scale horizontally; client caches (The client-side routing cache, as it cannot handle the increased number of shards efficiently) can be sharded or refreshed incrementally.

  2. Step 2: Analyze hash ring management at scale

    Hash ring management (The hash ring management service, because updating and propagating ring changes becomes a bottleneck) requires updating and propagating ring changes to all nodes; at 10x scale, this coordination overhead and update frequency cause it to become the first bottleneck.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Hash ring updates bottleneck first at large scale [check]
Hint: Hash ring updates bottleneck first at large scale [OK]
Common Mistakes:
5. At 10x scale, which Pastebin component is most likely to fail first and why?
hard
A. The relational database, because complex joins for analytics become a bottleneck
B. The analytics event ingestion pipeline, because Kafka partitions saturate and cause backpressure
C. The Expiration Worker, because scanning all snippets for TTL expiry becomes too slow
D. The snippet storage layer, because key-value stores cannot shard keys efficiently at scale

Solution

  1. Step 1: Analyze scaling bottlenecks

    Expiration Worker scans all snippets to delete expired ones; at 10x scale, scan latency grows linearly.

  2. Step 2: Compare other components

    Relational DB (A) is not primary storage; key-value stores (C) shard well; Kafka (D) scales with partitioning but expiration scanning is a linear bottleneck.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Expiration scanning is linear and slows down first at scale [check]
Hint: Expiration scanning scales linearly, fails first at 10x [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard