Bird
Raised Fist0

Design an E-Learning Platform - Course, Video, Progress, Certificate at Scale

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/7
API Gateway

Receives user request for course video and progress data

User requests video playback and current progress via HTTPS

Protocol:HTTPS
BeforeUser has not requested video or progress data
AfterAPI Gateway has received and parsed the request
Progress Service

Fetches current progress and sends progress update event

Reads progress from Cassandra and emits Kafka event if updated

Protocol:Internal RPC + Kafka
BeforeProgress Service has no recent progress event for user
AfterProgress Service has emitted progress update event to Kafka
Apache Kafka

Queues progress update event for downstream processing

Durably stores progress update event for consumption

Protocol:Kafka
BeforeKafka has no new progress event for this user
AfterKafka has durably stored the progress update event
Apache Cassandra

Persists updated user progress data

Writes progress update from Kafka consumer to Cassandra

Protocol:CQL (Cassandra Query Language)
BeforeCassandra has previous progress data for user
AfterCassandra has latest progress data persisted
Certificate Service

Checks progress to determine certificate eligibility

Queries Cassandra and updates certificate status in Elasticsearch

Protocol:CQL + Elasticsearch API
BeforeCertificate Service has outdated certificate status
AfterCertificate Service has updated certificate status in ES
Video CDN

Streams requested video content to user

Serves video chunks from cache or MinIO backend via HTTP Range

Protocol:HTTP Range
BeforeUser has not received any video data
AfterUser receives video stream chunks from CDN
MinIO (S3-compatible object store)

Serves video file chunks to Video CDN on cache miss

Reads video data from object storage for CDN

Protocol:S3 API
BeforeMinIO stores raw video files
AfterMinIO has served requested video chunks to CDN
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────┐
│  API Gateway  │
└───────┬───────┘
        │ HTTPS Request: Get Video + Progress
        ▼
┌───────────────┐
│ Progress Srv  │
└───────┬───────┘
        │ Kafka Event: Progress Update
        ▼
┌───────────────┐
│ Apache Kafka  │
└───────┬───────┘
        │ Cass. Write: Persist Progress
        ▼
┌───────────────┐
│ Apache Cass.  │
└───────┬───────┘
        │ Query: Check Certificate Eligibility
        ▼
┌───────────────┐
│ Certificate Srv│
└───────┬───────┘
        │ Elasticsearch Update: Certificate Status
        ▼
┌───────────────┐
│ Elasticsearch │
└───────────────┘
        ▲
        │ Video Request (HTTP Range)
┌───────┴───────┐
│  Video CDN    │
└───────┬───────┘
        │
┌───────┴───────┐
│ MinIO (S3) Obj│
└───────────────┘
Design Rationale

This architecture shows a vertically stacked flow for an e-learning platform. The API Gateway receives user requests for video playback and progress. The Progress Service fetches and updates progress, emitting events to Apache Kafka for durability and asynchronous processing. Kafka queues progress updates which are persisted in Apache Cassandra, a scalable NoSQL database. The Certificate Service queries Cassandra to check if the user qualifies for a certificate and updates Elasticsearch for fast retrieval. Video content is served via a Video CDN that caches video chunks, fetching from MinIO object storage on cache misses. This design decouples video delivery from progress tracking and certificate management, enabling scalability and fault tolerance.

Primary Focus

✓

Practice

(1/5)
1. A search ranking system must store and serve feature vectors for millions of users with low latency. Which component is best suited to provide fast, consistent feature retrieval at scale?
easy
A. A relational database with complex joins to compute features on demand
B. A distributed in-memory feature store with precomputed feature vectors
C. A batch processing system that recomputes features daily and stores them in object storage
D. A key-value store without versioning or consistency guarantees

Solution

  1. Step 1: Identify feature retrieval requirements

    Low latency and consistency for millions of users require fast, precomputed feature access.

  2. Step 2: Evaluate component suitability

    A relational DB with joins (A) is too slow for real-time; batch recomputation (C) is stale; key-value store without consistency (D) risks stale or inconsistent features. Distributed in-memory feature store (B) provides fast, consistent retrieval.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    In-memory feature store -> low latency, consistent features [check]
Hint: Precompute features in-memory for fast, consistent retrieval [check]
Common Mistakes:
2. A web application experiences frequent database read spikes causing high latency. Which component best reduces this latency by serving repeated reads efficiently?
easy
A. Implement an in-memory cache layer like Redis to serve repeated reads
B. Use a relational database with read replicas to distribute load
C. Add more application servers to handle increased traffic
D. Use a message queue to buffer incoming read requests

Solution

  1. Step 1: Identify the latency cause

    High latency is due to repeated database reads causing load spikes, which read replicas help but still hit the database.

  2. Step 2: Recognize caching benefits

    An in-memory cache like Redis serves repeated reads from memory, drastically reducing latency and database load.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Cache serves repeated reads faster than replicas [check]
Hint: Cache repeated reads to avoid DB spikes [check]
Common Mistakes:
3. If the API Gateway's rate limiting cache fails and is not replaced, what is the immediate failure mode observed by users?
medium
A. Users receive explicit rate limit errors even when under limit
B. Requests bypass rate limiting, causing backend overload and degraded service
C. The system automatically switches to database-level rate limiting with no impact
D. All requests are silently dropped without response

Solution

  1. Step 1: Understand cache failure impact

    Without the cache, the API Gateway cannot track usage accurately, often defaulting to conservative rate limit errors to protect backend.

  2. Step 2: Evaluate other options

    Silent drops cause user confusion; bypassing limits risks overload; automatic fallback to DB rate limiting is rare and slow.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Cache failure -> conservative rate limit errors -> user impact [check]
Hint: Cache failure triggers conservative rate limit errors [check]
Common Mistakes:
4. At 10x scale, which component in the notification system is most likely to fail first and why?
hard
A. The Delivery Workers, because they cannot horizontally scale beyond a fixed number of threads per machine
B. The push notification gateway, because mobile platform rate limits throttle outgoing messages
C. The relational database storing user preferences, due to increased read/write contention
D. The presence cache, because it is stateful and cannot be sharded easily across many servers

Solution

  1. Step 1: Analyze Delivery Worker scalability

    Delivery Workers are CPU and thread-bound; increasing scale requires more workers, but thread limits and resource contention cause bottlenecks.

  2. Step 2: Compare other components

    Relational DB (A) can be sharded; push gateways (C) are rate-limited but can be scaled with multiple credentials; presence cache (D) can be sharded with consistent hashing.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Delivery Workers hit thread and CPU limits first at 10x scale [check]
Hint: Delivery Workers hit thread limits before other components [check]
Common Mistakes:
5. At 10x scale, which component in the real-time analytics pipeline is most likely to fail first and why?
hard
A. Kafka brokers, due to partition leader overload and disk I/O bottlenecks
B. Dashboard frontend, as WebSocket connections exceed browser limits
C. Flink cluster, because stateful operators require large memory and checkpointing slows down
D. OLAP store, because complex ad-hoc queries overwhelm its indexing and caching layers

Solution

  1. Step 1: Analyze Kafka broker bottlenecks at scale

    Kafka brokers face partition leader overload and disk I/O saturation as event volume grows 10x.

  2. Step 2: Compare other components

    Flink scales with more nodes; OLAP stores can shard; dashboards scale horizontally; Kafka's broker disk and network limits are hardest to scale first.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Kafka broker disk I/O saturates first at 10x event volume [check]
Hint: Kafka broker disk I/O saturates before others scale [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard