Bird
Raised Fist0

Design YouTube - Upload, Transcoding Pipeline, CDN Delivery, View Count

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/7
Upload Service

Receive video upload

User uploads raw video file via HTTP POST

Protocol:HTTP POST
BeforeUser has video file ready to upload
AfterUpload Service received video and metadata
Kafka Upload Topic

Publish upload metadata

Upload Service publishes video metadata and reference to Kafka

Protocol:Kafka
BeforeUpload Service holds video metadata and reference
AfterMetadata durably stored in Kafka topic for processing
Transcoding Service

Consume upload metadata and transcode

Reads Kafka message, downloads raw video, transcodes formats

Protocol:Kafka Consumer + HTTP GET
BeforeTranscoding Service idle, Kafka topic has new message
AfterTranscoded video files ready for storage
Object Storage (GCS/S3)

Store transcoded video

Transcoding Service uploads video chunks to object storage

Protocol:HTTP PUT
BeforeTranscoded video files exist locally
AfterVideo files durably stored in Object Storage
CDN Edge Nodes

Serve video to viewers

CDN fetches video from Object Storage and streams to users

Protocol:HTTP GET
BeforeVideo stored in Object Storage, CDN cache cold
AfterVideo cached at CDN edges, viewers receive stream
View Count Service

Increment view count

Receives view event, updates count in NoSQL DB

Protocol:gRPC or REST
BeforeView count is last persisted value
AfterView count incremented and persisted in Cassandra
NoSQL DB (Cassandra)

Persist updated view count

Stores updated count with high write throughput

Protocol:CQL (Cassandra Query Language)
BeforeView count stored at previous value
AfterView count updated for video in database
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────┐
│   Upload Service  │
└─────────┬─────────┘
          │ HTTP POST (video file)
          ▼
┌───────────────────────┐
│  Kafka Upload Topic   │
└─────────┬─────────────┘
          │ Kafka Message (video metadata + ref)
          ▼
┌───────────────────────┐
│  Transcoding Service   │
└─────────┬─────────────┘
          │ Store transcoded video
          ▼
┌────────────────────────────┐
│ Object Storage (GCS/S3)    │
└─────────┬──────────────────┘
          │ HTTP PUT (video chunks)
          ▼
┌───────────────────┐
│ CDN Edge Nodes    │
└─────────┬─────────┘
          │ HTTP GET (video stream)
          ▼
┌───────────────────────┐
│ View Count Service    │
└─────────┬─────────────┘
          │ Update view count
          ▼
┌───────────────────────┐
│ NoSQL DB (Cassandra)  │
└───────────────────────┘
Design Rationale

This architecture shows the end-to-end flow of a video upload on YouTube. The Upload Service receives the raw video and metadata, which it publishes to a Kafka topic to decouple ingestion from processing. The Transcoding Service consumes Kafka messages, transcodes the video into multiple formats, and stores the results in durable Object Storage like GCS or S3. CDN Edge Nodes then serve the video streams to viewers with low latency by caching content closer to users. Meanwhile, the View Count Service tracks video views and updates counts in a high-throughput NoSQL database such as Cassandra. This design ensures scalability, durability, and responsiveness by separating concerns and using asynchronous messaging and caching layers.

Primary Focus

✓

Practice

(1/5)
1. During a flash sale, the system must handle sudden bursts of 10x traffic within 60 seconds, ensuring order preservation of purchase requests per user session. Which component best handles this requirement?
easy
A. Use a relational database with row-level locking to serialize requests
B. Use Redis as a cache to store purchase requests temporarily
C. Use Kafka with partitioning keyed by user session ID to preserve order
D. Use a simple HTTP load balancer to distribute requests evenly

Solution

  1. Step 1: Identify ordering requirement per user session

    Kafka partitions keyed by user session ID guarantee ordering within that partition, ensuring requests are processed sequentially per user.

  2. Step 2: Evaluate alternatives

    Relational DB row-level locking (A) causes high contention and poor scalability under 10x burst. Redis (C) lacks strong ordering guarantees and durability. Load balancer (D) only distributes load but does not preserve order.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Kafka partition key -> per-session ordering guaranteed [check]
Hint: Kafka partitions preserve order per key [check]
Common Mistakes:
2. In a system generating KSUIDs, at step 4 the ID generator component appends a random payload after the timestamp. What is the state of the generated ID at this step?
easy
A. The ID is a UUIDv4 embedded inside the KSUID format
B. The ID is fully deterministic and only contains the timestamp
C. The ID contains a timestamp prefix followed by a cryptographically secure random payload
D. The ID is a sequential number incremented per node

Solution

  1. Step 1: Understand KSUID structure at step 4

    KSUIDs combine a 4-byte timestamp prefix with a 16-byte random payload to ensure uniqueness and sortability.

  2. Step 2: Eliminate incorrect states

    Deterministic timestamp only (A) lacks uniqueness; UUIDv4 (C) is unrelated; sequential number (D) is not part of KSUID design.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    KSUID = timestamp + random payload for uniqueness and sortability [check]
Hint: KSUID = timestamp prefix + random payload [check]
Common Mistakes:
3. For a CDN serving frequently updated dynamic content, which origin content delivery strategy is best to ensure freshness without overloading the origin server?
medium
A. Pull model where edge nodes fetch content from origin on cache miss with TTL-based caching
B. Push model where origin proactively sends all content updates to edge nodes
C. Push model with edge nodes polling origin periodically for updates
D. Pull model with edge nodes never caching content and always fetching from origin

Solution

  1. Step 1: Analyze push vs pull for dynamic content freshness

    Push model can overload origin with frequent updates; polling adds unnecessary load.

  2. Step 2: Evaluate pull with TTL caching

    Pull with TTL balances freshness and origin load by caching briefly and refreshing on expiry.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Pull with TTL caching balances freshness and origin load [check]
Hint: Pull with TTL balances freshness and load [check]
Common Mistakes:
4. At 10x scale, which component of a Bloom filter system is most likely to fail first and why?
hard
A. The client application, due to increased latency in membership queries
B. The bit array storage, because increased insertions cause saturation and higher false positive rates
C. The hash functions, because their computation time grows linearly with scale
D. The network bandwidth, due to Bloom filter synchronization overhead

Solution

  1. Step 1: Identify scaling impact on Bloom filter components

    At very high scale, hash function computation cost increases linearly with insertions, becoming a CPU bottleneck.

  2. Step 2: Evaluate other components

    Bit array saturation affects false positives but does not cause immediate failure; network and client latency scale better.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Hash function CPU cost grows linearly, causing first failure at scale [check]
Hint: Hash function CPU cost grows linearly at scale [check]
Common Mistakes:
5. At 10x scale, which component in Netflix's adaptive streaming pipeline is most likely to fail first and why?
hard
A. Personalization service because recommendation model retraining becomes too slow
B. Origin servers because storage I/O bandwidth saturates under high request volume
C. CDN edge servers because WebSocket connections for adaptive bitrate are stateful and cannot be easily sharded
D. Client devices because decoding multiple bitrate streams consumes excessive CPU

Solution

  1. Step 1: Identify stateful bottlenecks at scale

    Personalization retraining (Personalization service because recommendation model retraining becomes too slow) can be scaled horizontally with distributed training. Origin storage (Origin servers because storage I/O bandwidth saturates under high request volume) can be scaled with sharding and CDN caching. Client devices (Client devices because decoding multiple bitrate streams consumes excessive CPU) are outside server control.

  2. Step 2: Analyze CDN edge server scaling challenges

    WebSocket connections for adaptive streaming are stateful and bound to specific edge servers, limiting horizontal scaling and resharding, causing edge server overload at 10x scale.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Stateful WebSocket limits CDN edge server scalability [check]
Hint: Stateful WebSocket limits CDN edge scaling first [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard