Bird
Raised Fist0

Design Facebook News Feed - EdgeRank, Fan-Out on Write vs Read

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
▶
Request Trace
Step 1/6
API Gateway

Receives new post request

User submits a new status update via HTTP POST

Protocol:HTTP POST
BeforeUser has composed a new post on client device
AfterAPI Gateway forwards post to Write Service
Write Service (Post Ingest)

Processes and writes post to Kafka

Validates and serializes post, produces to Kafka topic

Protocol:Kafka Produce
BeforePost received but not durably stored
AfterPost durably appended to Kafka write log
Fan-out Workers

Consume post from Kafka and fan-out

Reads new post, identifies followers, writes to feed storage and cache

Protocol:Kafka Consume, Cassandra Write, Redis SET
BeforePost is in Kafka but not yet distributed
AfterFollowers' feeds updated in Cassandra and Redis cache
Feed Storage (Cassandra)

Stores post in followers' feed partitions

Writes post data keyed by follower ID for efficient retrieval

Protocol:Cassandra Write
BeforeFollowers' feeds do not contain the new post
AfterFollowers' feeds persistently updated with new post
Cache Layer (Redis)

Caches recent feed entries

Sets new post in Redis cache for fast read access

Protocol:Redis SET
BeforeCache does not contain the new post
AfterCache updated with new post for quick retrieval
Read Service (Feed Retrieval)

Serves feed request from cache or storage

Reads user feed from Redis cache; falls back to Cassandra if miss

Protocol:Redis GET / Cassandra Query
BeforeUser requests feed, cache may or may not have data
AfterUser receives up-to-date feed including new post
🏗️
Architecture Diagram
Architecture Diagram
┌───────────────────────────┐
│       API Gateway         │
└─────────────┬─────────────┘
              │ HTTP POST
              ▼
┌───────────────────────────┐
│ Write Service (Post Ingest)│
└─────────────┬─────────────┘
              │ Kafka Produce
              ▼
┌───────────────────────────┐
│    Kafka (Write Log)      │
└─────────────┬─────────────┘
              │ Kafka Consume
              ▼
┌───────────────────────────┐
│     Fan-out Workers       │
└───────┬─────────┬─────────┘
        │         │
        │         │
        │         │
        │         │
        ▼         ▼
┌───────────────┐ ┌─────────────────────┐
│Feed Storage   │ │  Cache Layer (Redis) │
│ (Cassandra)   │ │                     │
└───────┬───────┘ └─────────┬───────────┘
        │                   │ Redis SET
        ▼                   ▼
┌───────────────────────────┐
│ Read Service (Feed Retrieval)│
└─────────────┬─────────────┘
              │ Redis GET / Cassandra Query
              ▼
       (User's News Feed)
Design Rationale

This architecture uses fan-out on write to push new posts to followers' feeds immediately after ingestion. The API Gateway receives the post and forwards it to the Write Service, which appends it durably to Kafka. Fan-out Workers consume from Kafka, identify followers, and write the post to both Cassandra for persistence and Redis for fast cache access. The Read Service serves user feed requests by first checking Redis cache, falling back to Cassandra if needed. This design optimizes read latency at the cost of write amplification, ensuring users see fresh feeds quickly. The stepwise trace highlights how data flows through each component, emphasizing durability, fan-out, and caching roles.

Primary Focus

✓

Practice

(1/5)
1. In the rate limiter request flow, at Step 4 the Distributed Counter service receives an increment request. What happens if the counter exceeds the configured limit at this step?
easy
A. The request is immediately rejected and an error is returned to the client
B. The counter increments anyway but a warning is logged for later analysis
C. The increment is delayed until the next time window resets the counter
D. The request is forwarded to a fallback service that applies a different limit

Solution

  1. Step 1: Trace Distributed Counter behavior at Step 4

    The counter checks if the increment would exceed the limit; if yes, it rejects the request immediately.

  2. Step 2: Eliminate incorrect states

    Allowing increment with warning (B) breaks enforcement; delaying increment (C) causes inconsistent state; forwarding (D) adds complexity and is not standard.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Counter rejects requests exceeding limit immediately [check]
Hint: Reject requests immediately when limit exceeded [check]
Common Mistakes:
2. In a system where messages are produced to Kafka and consumed by a delivery worker, at step 4 the Delivery Worker reads the message offset from Kafka. What state does the Delivery Worker mark if the downstream consumer acknowledges receipt?
easy
A. Marks the message as sent but does not commit the offset yet
B. Deletes the message from Kafka immediately
C. Marks the message as delivered and commits the offset to Kafka
D. Retries sending the message regardless of acknowledgment

Solution

  1. Step 1: Understand Kafka offset commit semantics

    Kafka consumers commit offsets only after processing messages successfully to avoid reprocessing.

  2. Step 2: Delivery Worker behavior on acknowledgment

    Upon downstream acknowledgment, the worker marks the message as delivered and commits the offset, signaling safe processing.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Offset commit after acknowledgment ensures exactly-once processing [check]
Hint: Commit Kafka offset only after downstream ack to avoid duplicates [check]
Common Mistakes:
3. If the API Gateway's authentication service crashes, what is the immediate failure mode observed by clients attempting to access protected endpoints?
medium
A. Clients receive a 500 Internal Server Error due to gateway crash
B. Clients receive a 401 Unauthorized response because authentication cannot be verified
C. Requests bypass authentication and reach backend services unverified
D. Requests queue indefinitely at the gateway until authentication service recovers

Solution

  1. Step 1: Trace failure impact of authentication service crash

    Without the authentication service, the gateway cannot verify credentials and may default to allowing requests through if no fail-closed mechanism is implemented.

  2. Step 2: Evaluate other options

    Clients receive a 500 Internal Server Error due to gateway crash is unlikely as gateway remains up. Clients receive a 401 Unauthorized response because authentication cannot be verified requires fail-closed design, which is not always implemented. Requests queue indefinitely at the gateway until authentication service recovers causes indefinite blocking, which is rare.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Auth service down -> requests pass unverified if no fail-closed [check]
Hint: Auth crash -> requests may bypass if no fail-closed design [check]
Common Mistakes:
4. If the reverse proxy responsible for SSL termination crashes, what exactly breaks in the request flow?
medium
A. Clients can still connect directly to backend servers without interruption
B. Clients' SSL handshakes fail, so connections never reach backend servers
C. Backend servers receive unencrypted traffic, exposing internal data
D. The reverse proxy forwards requests without decrypting, causing backend errors

Solution

  1. Step 1: Understand reverse proxy SSL termination failure

    If the reverse proxy crashes, SSL termination stops, so backend servers receive unencrypted traffic if clients connect directly or fallback occurs.

  2. Step 2: Eliminate other options

    Clients cannot connect directly if proxy is mandatory; SSL handshake fails only if proxy blocks connection; proxy can't forward encrypted requests without decrypting if crashed.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Reverse proxy crash exposes backend to unencrypted traffic [check]
Hint: Reverse proxy crash exposes backend to unencrypted traffic [check]
Common Mistakes:
5. If the gRPC server's message queue is removed, what exactly breaks in the message delivery flow?
medium
A. Clients receive duplicate messages due to lack of deduplication
B. The server blocks sending until the client acknowledges, causing backpressure
C. Messages are lost if the client is slow or disconnected, with no retry mechanism
D. The server switches to synchronous unary calls, increasing latency

Solution

  1. Step 1: Role of message queue in gRPC server

    The queue buffers messages for slow or disconnected clients, enabling retries and flow control.

  2. Step 2: Effect of removing the queue

    Without the queue, messages sent to slow/disconnected clients are dropped immediately, causing message loss without retries.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    No queue -> message loss on slow/disconnected clients [check]
Hint: No queue -> messages lost if client slow or disconnected [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard