Bird
Raised Fist0

Design Quora - Q&A Feed, Search, Ranking, Spam Detection, Follow Topics

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/8
Write Frontend Servers

User submits a new question post

Frontend validates input and sends HTTP request to API Gateway

Protocol:HTTP/REST
BeforeUser has composed a new question post on frontend
AfterPost data sent as HTTP request to API Gateway
API Gateway

Receives post request and forwards to Spam Detection Service

Performs authentication and routes request for spam analysis

Protocol:REST
BeforeAPI Gateway has incoming HTTP request with post data
AfterPost data forwarded to Spam Detection Service for analysis
Spam Detection Service

Analyzes post content for spam and flags if necessary

Returns spam verdict to API Gateway

Protocol:REST
BeforeSpam Detection Service has raw post content to analyze
AfterSpam verdict returned; post marked clean or flagged
Kafka (Write Event Bus)

API Gateway publishes post event to Kafka topic

Event includes post data and spam status for downstream processing

Protocol:Kafka Publish
BeforePost approved by spam detection, ready for persistence
AfterPost event durably stored in Kafka topic
Cassandra (Posts DB)

Feed Generation Service consumes Kafka event and stores post

Post data persisted in Cassandra for durable storage

Protocol:CQL
BeforeFeed Generation Service has new post event from Kafka
AfterPost data stored in Cassandra database
Elasticsearch (Search Index)

Feed Generation Service indexes post for search

Post content sent to Elasticsearch via bulk REST API

Protocol:REST Bulk API
BeforePost stored in Cassandra, ready for search indexing
AfterPost indexed in Elasticsearch for fast search queries
Feed Generation Service

Generates personalized feed including new post

Updates user feed data and caches in Redis

Protocol:Redis SET
BeforePost indexed and available for feed generation
AfterUser feed updated and cached in Redis for fast access
Redis (Cache Layer)

Caches generated feed for quick retrieval by frontend

Feed data stored with TTL for freshness

Protocol:Redis GET/SET
BeforeFeed Generation Service has updated feed data
AfterFeed cached in Redis, ready for frontend queries
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────┐
│  Write Frontend Srv   │
│  (User submits post)  │
└─────────────┬─────────┘
              │ HTTP/REST
              ▼
┌───────────────────────┐
│     API Gateway       │
│ (Auth, Routing, QoS)  │
└─────────────┬─────────┘
              │ REST
              ▼
┌───────────────────────────┐
│  Spam Detect Svc          │
│ (Analyze content, flag sp)│
└─────────────┬─────────────┘
              │ REST
              ▼
┌───────────────────────────┐
│ Kafka (Write Event Bus)   │
│ (Durable event streaming) │
└─────────────┬─────────────┘
              │ Kafka Publish
              ▼
┌───────────────────────────┐
│ Cassandra (Posts DB)      │
│ (Store post data durably) │
└─────────────┬─────────────┘
              │ CQL
              ▼
┌────────────────────────────┐
│ Elasticsearch (Search Ind) │
│ (Index post for search)    │
└─────────────┬──────────────┘
              │ REST Bulk API
              ▼
┌───────────────────────────┐
│ Feed Gen Svc              │
│ (Generate personalized fd)│
└─────────────┬─────────────┘
              │ Redis SET
              ▼
┌───────────────────────────┐
│ Redis (Cache Layer)       │
│ (Cache user feeds)        │
└───────────────────────────┘
Design Rationale

This architecture shows a new question post flowing from the user frontend through the API Gateway, which handles authentication and routes the request to the Spam Detection Service. After spam analysis, the post event is published to Kafka, ensuring durable and ordered event streaming. The Feed Generation Service consumes Kafka events, storing posts in Cassandra for durability and indexing them in Elasticsearch for search capabilities. It then generates personalized feeds, caching them in Redis for low-latency retrieval. This vertical flow highlights separation of concerns, asynchronous event-driven processing, and caching to optimize read performance and scalability.

Primary Focus

✓

Practice

(1/5)
1. In the message delivery trace, at Step 4 the Delivery Worker checks Bob's presence TTL in the presence service cache. What happens if Bob's presence TTL has expired at this step?
easy
A. The Delivery Worker queues the message for offline storage and does not send over WebSocket
B. The Delivery Worker immediately marks the message as delivered without sending
C. The Delivery Worker retries presence check indefinitely before sending
D. The Delivery Worker sends the message over WebSocket regardless of presence TTL

Solution

  1. Step 1: Trace Delivery Worker decision at Step 4 -- Bob's presence TTL

    If Bob's presence TTL expired, Delivery Worker treats Bob as offline and queues message for offline storage.

  2. Step 2: Eliminate other options

    Marking delivered without sending is incorrect; retrying indefinitely blocks delivery; sending regardless ignores offline status.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Expired presence TTL -> queue for offline delivery [check]
Hint: Expired presence TTL means offline -> queue message [check]
Common Mistakes:
2. A system must handle 1 million writes per second and fan-out to millions of users asynchronously. Which component best handles decoupling write load from delivery?
easy
A. Use direct HTTP calls from sender to all receivers to guarantee delivery order
B. Use a relational database with sharded writes to handle all writes and fan-out synchronously
C. Use an in-memory cache like Redis to store all messages and fan-out immediately
D. Use a distributed message queue like Kafka to buffer writes and enable asynchronous fan-out

Solution

  1. Step 1: Identify the need for decoupling writes from fan-out

    Handling 1 million writes per second with synchronous fan-out is not feasible on a single machine or direct calls.

  2. Step 2: Evaluate components for asynchronous buffering

    Kafka provides durable, partitioned logs that buffer writes and allow scalable asynchronous delivery. Relational DBs can't handle this write load synchronously. Redis lacks durability guarantees for this scale. Direct HTTP calls block and don't scale.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka buffers writes asynchronously, enabling high write throughput [check]
Hint: Use durable queues to decouple high write load [check]
Common Mistakes:
3. Without a message queue like Kafka in a chat system, what exactly breaks in message delivery?
medium
A. Alice's send blocks until Bob's server responds; if Bob's server crashes, message is lost with no retry
B. Messages are duplicated due to lack of idempotency
C. Messages are encrypted incorrectly causing security breaches
D. Delivery workers lose track of message ordering causing out-of-order display

Solution

  1. Step 1: Role of Kafka in decoupling send and delivery

    Kafka buffers messages asynchronously, preventing sender blocking on recipient availability.

  2. Step 2: Consequence of removing Kafka

    Sender must synchronously wait for recipient response; if recipient crashes, message is lost with no retry.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    No queue causes synchronous blocking and message loss on recipient failure [check]
Hint: No queue means sender blocks and loses messages if recipient crashes [check]
Common Mistakes:
4. If the Kafka cluster responsible for connection event streaming goes down, what exactly breaks in the system's connection request processing?
medium
A. Connection requests are lost immediately with no retry or buffering
B. Connection requests are redirected to a backup database that duplicates all writes
C. Connection requests block the user synchronously until Kafka recovers, causing UI timeouts
D. Connection requests are accepted but downstream services do not receive updates, causing stale connection graphs

Solution

  1. Step 1: Understand Kafka's role in event streaming

    Kafka buffers and reliably delivers connection events to downstream services asynchronously.

  2. Step 2: Trace failure impact

    If Kafka is down, writes to Kafka fail but connection requests may still be accepted at the API layer; however, downstream consumers do not get updates, causing stale graph state.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Kafka down -> downstream stale, no immediate request loss [check]
Hint: Kafka failure stalls downstream, not immediate request loss [OK]
Common Mistakes:
5. At 10x scale, which component fails first in a real-time chat system and why?
hard
A. The database fails first due to increased write volume causing slow queries
B. The message queue fails first due to partition leader overload
C. The load balancer fails first because it cannot handle the increased TCP connections
D. The chat servers fail first because WebSocket connections are stateful and cannot be resharded easily

Solution

  1. Step 1: Identify stateful bottleneck at scale

    Chat servers maintain stateful WebSocket connections bound to server memory, making resharding difficult at 10x scale.

  2. Step 2: Compare other components

    Databases (A) can be scaled with sharding and caching; message queues (C) can add partitions; load balancers (D) scale horizontally more easily.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Stateful WebSocket servers -> first bottleneck at 10x scale [check]
Hint: Stateful WebSocket servers block scaling; resharding is hard [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard