Bird
Raised Fist0

Spotify - Audio Streaming, Playlist, Offline Download Architecture

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🏗️
Spotify - Audio Streaming, Playlist, Offline Download Architecture
Design a scalable, low-latency audio streaming platform with playlist management and offline download support for 2B DAU
hardVideo and Media StreamingFreq: high
Prerequisites:System Design FundamentalsCAP TheoremDistributed Storage SystemsContent Delivery NetworksData Modeling for High Scale
⚙️
Functional Requirements
  • Stream audio tracks on-demand with adaptive bitrate support
  • Create, update, and delete user playlists with millions of users concurrently
  • Support offline download of playlists and tracks with DRM enforcement
  • Search and browse music catalog metadata
  • Sync user playlists and offline content across devices
  • Provide personalized recommendations and playback history
📊
Non-Functional Requirements
  • Support 2 billion daily active users (DAU)
  • 99.99% availability for streaming and playlist services
  • 100ms P99 latency for playlist CRUD and playback start
  • Handle peak write throughput of 925K writes/sec (playlist updates, downloads)
  • Efficient bandwidth usage with adaptive streaming and CDN caching
❓
Clarifying Questions to Ask
  • What is the expected average track length and bitrate for streaming?
  • How many tracks per playlist on average and max expected size?
  • What is the offline download retention policy and storage limits per user?
  • Is DRM required for offline downloads and how strict is enforcement?
  • What are the peak concurrency and geographic distribution of users?
  • Are playlists shared publicly or only private to users?
Out of Scope
✗ Music recommendation algorithms and ML models
✗ User authentication and social features
✗ Payment and subscription management
✗ Video streaming or podcast support
📌
Assumptions
  • 2 billion DAU
  • Average 40 playlist-related writes per user per day (create, update, delete, download)
  • Average playlist size 50 tracks, average track size 5MB (320kbps, 3min)
  • Average track streamed 1.5 times per user per day
  • Retention of offline downloads: 7 days
⚡
QPS Calculation
write qps2B × 40 / 86400 ≈ 925,926 writes/sec
peak write qpsAssuming 3x peak factor: 2.78M writes/sec
read qps2B × 1.5 (streams) / 86400 ≈ 34,722,222 reads/sec
💾
Storage Estimate
daily storage2B users × 40 writes × 100 bytes/metadata write = 8TB/day metadata
audio storage daily2B × 1.5 streams × 5MB = 15PB/day (streamed, not stored)
offline storageAssuming 10% users download 50 tracks × 5MB = 25GB/user × 200M users = 5EB total
retention storage5EB × 7 days = 35EB total offline storage
with replicationMetadata 3x replication = 24TB/day, Offline 3x = 105EB
🖥️
Server Estimate
metadata serviceAssuming 10K writes/sec per server, need 278 servers for peak writes
streaming edge serversAssuming 1Gbps per server, need ~5200 servers for peak streaming bandwidth
🏗️
Architecture Diagram
Architecture Diagram
          +----------------+
          |  User Devices  |
          +-------+--------+
                  |
          +-------v--------+
          |  CDN (Akamai)  |
          +-------+--------+
                  |
          +-------v--------+
          |  Streaming Svc |
          +-------+--------+
                  |
          +-------v--------+
          | Playlist Svc   |
          +-------+--------+
                  |
          +-------v--------+
          | Metadata Store |
          +-------+--------+
                  |
          +-------v--------+
          |  Offline Svc   |
          +-------+--------+
                  |
          +-------v--------+
          |  Object Store  |
          +----------------+
🧩
Components
Content Delivery Network (Akamai)
Cache and deliver audio streams globally with low latency
Why thisAkamai provides a global edge network with optimized routing and adaptive bitrate support for audio streaming
Why notCloudFront lacks the global scale and specialized media optimizations needed for 2B DAU streaming
Streaming Service (Custom microservice)
Handles streaming session management, adaptive bitrate switching, and DRM enforcement
Why thisCustom service allows fine-grained control over streaming logic and integrates with DRM providers
Why notUsing generic HTTP servers (e.g., Nginx) lacks session and DRM logic, limiting control and security
Playlist Service (gRPC microservice)
Manages playlist CRUD operations, sync, and user metadata
Why thisgRPC enables low-latency, high-throughput RPC calls with strong typing and streaming support for playlist updates
Why notREST APIs add overhead and lack streaming capabilities needed for real-time sync
Metadata Store (Apache Cassandra)
Stores user playlists, track metadata, and offline download metadata with high write throughput
Why thisCassandra provides linear write scalability, tunable consistency, and multi-datacenter replication for 99.99% availability
Why notPostgreSQL requires complex manual sharding at 2.78M writes/sec; B-tree writes don't scale linearly
Offline Service (Custom microservice)
Manages offline download requests, DRM token issuance, and sync across devices
Why thisCustom logic needed to enforce DRM, track download status, and handle device sync securely
Why notUsing CDN or streaming service alone cannot enforce offline DRM or manage device sync state
Object Store (Amazon S3 or equivalent)
Stores encrypted audio files for streaming and offline download
Why thisS3 provides durable, scalable, and cost-effective storage with lifecycle policies and encryption support
Why notHDFS lacks global availability and high availability guarantees needed for user-facing streaming
🔄
Data Flow
1
User Device → Playlist Service
User creates/updates playlist
Playlist service validates and writes metadata to Cassandra
2
Playlist Service → Metadata Store (Cassandra)
Write playlist metadata
Cassandra handles high write throughput with tunable consistency
3
User Device → Streaming Service via CDN
User requests audio stream
CDN serves cached content or fetches from streaming service if cache miss
4
Streaming Service → Object Store
Fetch encrypted audio file
Streaming service retrieves audio from object store if not cached in CDN
5
User Device → Offline Service
User requests offline download
Offline service validates DRM and issues tokens
6
Offline Service → Object Store
Download encrypted audio files for offline storage
Offline service manages download and sync state
7
Offline Service → Metadata Store (Cassandra)
Update offline download metadata
Tracks download status and retention

Practice

(1/5)
1. You need a storage system for a database requiring low-latency random reads/writes and strong consistency. Which storage type is best and why?
medium
A. Block storage, because it provides low-latency random access and strong consistency guarantees.
B. Tape storage, because it is cost-effective for large data volumes.
C. File storage, because it offers hierarchical namespace and POSIX compliance.
D. Object storage, because it scales massively and supports HTTP APIs.

Solution

  1. Step 1: Identify storage needs for low-latency random access and strong consistency

    Databases require fast block-level access with strong consistency for transactional integrity.

  2. Step 2: Evaluate storage types

    Block storage offers raw device access with low latency and strong consistency; object storage is optimized for throughput and eventual consistency; file storage adds overhead and less suited for random writes; tape is offline archival.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Block storage = low latency + strong consistency [check]
Hint: Block storage = low latency + strong consistency for DBs [check]
Common Mistakes:
2. If the Bloom filter's hash functions are not independent and produce correlated outputs, what is the exact failure mode observed?
medium
A. The bit array will saturate quickly causing excessive false positives and reduced filter effectiveness
B. The filter will have false negatives, incorrectly rejecting existing members
C. The false positive rate increases beyond the expected theoretical rate
D. Insertions will fail silently without updating the bit array

Solution

  1. Step 1: Understand hash function independence role

    Independent hash functions distribute bits uniformly; correlated hashes cause clustering.

  2. Step 2: Trace effect of correlated hashes

    Correlated hashes set overlapping bits, saturating the bit array faster, increasing false positives and reducing filter effectiveness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Correlated hashes cause bit array saturation, raising false positives [check]
Hint: Correlated hashes cause bit array saturation and false positives [check]
Common Mistakes:
3. If the message queue (Kafka) is unavailable during the flash sale, what exactly breaks in the order processing pipeline?
medium
A. Payment service retries indefinitely, causing duplicate charges
B. Order requests block at the API layer, causing client timeouts and lost orders
C. Orders are accepted but payment processing is delayed indefinitely
D. Orders are accepted but inventory updates are lost, causing overselling

Solution

  1. Step 1: Trace Kafka's role in order processing

    Kafka decouples order acceptance from inventory update; if Kafka is down, inventory update messages are lost or not sent.

  2. Step 2: Analyze failure impact

    API layer (B) uses async calls, so requests don't block. Payment retries (D) are independent. Accepting orders without inventory update (C) causes overselling.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka down -> inventory updates lost -> overselling risk [check]
Hint: Kafka outage loses inventory updates causing overselling [check]
Common Mistakes:
4. If the Kafka cluster responsible for fan-out message delivery crashes, what is the exact failure observed in the News Feed system?
medium
A. New posts are lost permanently and never delivered to followers
B. The system blocks new post writes until Kafka recovers, causing user-facing delays
C. Followers receive duplicate feed updates due to Kafka replaying messages
D. Delivery Workers cannot fetch fan-out messages, so followers see stale feeds until recovery

Solution

  1. Step 1: Understand Kafka's role in fan-out delivery

    Kafka acts as a durable queue for fan-out messages. If it crashes, Delivery Workers cannot fetch new messages, causing stale feeds.

  2. Step 2: Evaluate failure modes

    Posts are not lost immediately (A) due to Kafka's durability; writes are asynchronous so no blocking (B); duplicates (D) occur only on replay, not crash.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka crash stalls delivery, causing stale feeds [check]
Hint: Kafka crash stalls delivery, causing stale feeds [check]
Common Mistakes:
5. At 10x current scale, which component is most likely to fail first in Instagram's real-time feed delivery, and why?
hard
A. The relational database storing user profiles, due to increased read/write contention
B. The feed cache layer, because cache invalidation traffic grows exponentially with user base
C. The WebSocket servers handling live feed updates, due to stateful connections limiting horizontal scaling
D. The photo storage service, because object storage bandwidth saturates under heavy upload load

Solution

  1. Step 1: Identify stateful components limiting scale

    WebSocket servers maintain persistent connections per user, which are stateful and bound to server memory.

  2. Step 2: Analyze scaling challenges

    At 10x scale, adding servers doesn't migrate existing WebSocket connections, causing uneven load and resource exhaustion. Relational DB (A) can be sharded, cache (B) can be scaled with TTL tuning, storage (D) scales with cloud object storage.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Stateful WebSocket connections limit horizontal scaling [check]
Hint: Stateful WebSocket servers bottleneck horizontal scaling [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard