Bird
Raised Fist0

Design a Generic Newsfeed System - Pull vs Push vs Hybrid Fan-Out at Any Scale

HLDhardInterview freq: high
Choose your preparation mode4 modes available
ArchitectureTraceChallenge
🎯
Design a Generic Newsfeed System - Pull vs Push vs Hybrid Fan-Out at Any Scale
45 min · senior
senior

You are designing a newsfeed system for a social media startup with 50 million daily active users (DAU). The system must deliver personalized feeds with a P99 latency under 200ms. Users follow millions of other users and pages, generating billions of posts daily. The system must support end-to-end encryption for user privacy, offline delivery of feed updates, and handle sudden spikes in traffic during viral events.

Constraints
  • End-to-end encryption required for all feed content at rest and in transit
  • Offline delivery support: users should receive missed feed updates upon reconnect
  • P99 feed load latency must be under 200ms even at peak load
  • Support 50M DAU with an estimated 5 billion feed updates per day
  • Handle sudden viral spikes with 10x traffic bursts gracefully
📋
Guided Steps
1/5
Step 1
Choose the Fan-Out Strategy: Pull, Push, or Hybrid
Given 50M DAU and billions of posts daily, which fan-out approach best balances latency, scalability, and storage?
💡 Hint
Consider how user follow graph size and write vs read amplification affect each approach.
Key Decision
Hybrid fan-out using Apache Kafka for write ingestion and Redis Streams for per-user feed queues balances write amplification and read latency.
Interviewer Expectation
Mid: Names pull and push approaches with basic pros/cons. Senior: Explains write amplification in push, read amplification in pull, and how hybrid offloads hot users to push while cold users use pull.
⚠️Common Wrong Answer
Choosing pure push with Cassandra for all users leads to massive write amplification and storage explosion at 50M DAU.
Step 2
Design Data Storage for Feed Content and Metadata
What storage technologies and data models will you use to store encrypted feed content and user feed metadata for efficient retrieval and updates?
💡 Hint
Think about write-heavy workloads, range queries for cursor-based pagination, and encryption at rest.
Key Decision
Use Cassandra with partition_key=user_id and clustering_key=timestamp for feed metadata; encrypted S3 for feed content blobs to offload large payload storage.
Interviewer Expectation
Mid: Suggests NoSQL for scalability. Senior: Explains Cassandra’s LSM-tree write optimization, partitioning by user_id for O(1) range scans, and offloading large encrypted content to S3 to reduce DB load.
⚠️Common Wrong Answer
Using relational DB for feed metadata causes hotspots and poor write throughput at scale.
Step 3
Implement Offline Delivery and Sync Mechanism
How will the system ensure users receive missed feed updates after being offline, while maintaining P99 < 200ms on reconnect?
💡 Hint
Consider cursor-based sync, durable message queues, and client acknowledgments.
Key Decision
Use Redis Streams per user with consumer groups to track delivery state and enable efficient cursor-based replay of missed feed items.
Interviewer Expectation
Mid: Mentions cursor or timestamp-based sync. Senior: Details Redis Streams consumer groups for durable, ordered delivery and how client ACKs update stream offsets to avoid duplicate delivery.
⚠️Common Wrong Answer
Stateless HTTP polling causes excessive load and high latency on reconnect.
Step 4
Ensure End-to-End Encryption and Secure Delivery
How will you implement end-to-end encryption for feed content while allowing efficient feed generation and delivery?
💡 Hint
Think about client-side encryption keys and how to handle encrypted blobs in storage and transit.
Key Decision
Clients encrypt feed content with user-specific keys before upload; backend stores encrypted blobs in S3 and metadata in Cassandra without decrypting, ensuring zero-trust storage.
Interviewer Expectation
Mid: Mentions encryption at rest and in transit. Senior: Explains client-side encryption key management, zero-trust backend storage, and how encrypted metadata enables feed ranking without decrypting content.
⚠️Common Wrong Answer
Encrypting only at transport layer exposes data at rest and backend services.
Step 5
Design API Endpoints for Posting, Fetching, and Acknowledging Feed Items
Sketch the 3 core API endpoints and their request/response semantics to support posting new content, fetching personalized feeds, and acknowledging delivery.
💡 Hint
Consider idempotency, pagination cursors, and offline sync support.
Key Decision
POST /feed/post with encrypted payload and metadata; GET /feed?cursor= for paginated feed fetch; POST /feed/ack to update delivery state and advance cursor.
Interviewer Expectation
Mid: Names endpoints and basic parameters. Senior: Details idempotency keys on post, cursor-based pagination with opaque tokens, and ACK semantics to update Redis Streams consumer offsets.
⚠️Common Wrong Answer
Using simple offset pagination without idempotency leads to inconsistent feed and duplicate items.
🔌
API Design Challenge
Sketch 3 core API endpoints for the newsfeed system that cover posting content, fetching the feed, and acknowledging feed delivery.
Hints
  • How does a client submit a new post or update to the system?
  • How does a client efficiently fetch their personalized feed with pagination or cursor?
  • How does the client acknowledge receipt or read status of feed items for offline sync?
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Clear explanation of fan-out trade-offs with hybrid approach rationale
  • Use of Cassandra partitioning for feed metadata to enable scalable range queries
  • Use of Redis Streams for offline delivery and durable message replay
  • Client-side encryption for end-to-end security
Good to Have - 30%
  • API design covering posting, fetching with cursor, and ACK for delivery
  • Handling viral spikes with autoscaling Kafka and Redis
  • Offloading large encrypted content to S3 to reduce DB load
Bonus - 10%
  • Discussion of zero-trust backend storage and encryption key management
  • Detailed failure mode analysis of fan-out strategies under viral load
Anti-patterns to Avoid
Pure push fan-out with Cassandra writes for all users causes massive write amplification and storage explosion, making the system unscalable at 50M DAU
Stateless HTTP polling for offline sync causes excessive load and high latency on reconnect, collapsing under heavy user reconnects
Using relational databases for feed metadata causes hotspots and poor write throughput, leading to system bottlenecks
Encrypting only in transit exposes data at rest and backend services to breaches, violating end-to-end encryption requirements
Simple offset pagination without idempotency leads to inconsistent feed results and duplicate items, degrading user experience
Ignoring autoscaling for Kafka and Redis clusters results in latency spikes and system failures during viral traffic bursts
✅
Model Answer
▼ Reveal

This design uses a hybrid fan-out approach combining Apache Kafka for ingesting posts and Redis Streams for per-user feed queues to balance write and read amplification. Feed metadata is stored in Cassandra partitioned by user_id and timestamp for efficient range queries, while large encrypted content blobs are offloaded to AWS S3. Client-side encryption ensures end-to-end security with zero-trust backend storage. Offline delivery is supported via Redis Streams consumer groups enabling cursor-based replay of missed feed items. The API includes idempotent POST /feed/post, paginated GET /feed with opaque cursors, and POST /feed/ack endpoints for delivery acknowledgment. Autoscaling Kafka and Redis clusters handle 10x viral traffic spikes gracefully.

Key Decisions
  • Use Apache Kafka for scalable, ordered ingestion of posts keyed by user_id to reduce write amplification
  • Implement Redis Streams per user with consumer groups to track delivery state and enable efficient offline sync with cursor-based replay
  • Store feed metadata in Cassandra partitioned by user_id and clustered by timestamp to enable O(1) range scans for pagination
  • Offload large encrypted feed content blobs to AWS S3 to reduce database load and enable zero-trust storage
  • Enforce client-side encryption with user-specific keys to ensure end-to-end encryption and prevent backend decryption
  • Design API endpoints with idempotent POST /feed/post using idempotency keys, GET /feed with opaque cursor tokens for pagination, and POST /feed/ack to update delivery state and advance consumer offsets
  • Autoscale Kafka and Redis clusters dynamically to handle sudden 10x viral traffic spikes without latency degradation
  • Apache Kafka for ingesting posts with partition key=user_id to ensure ordering and scalability
  • Redis Streams per user for durable, ordered feed delivery and offline sync with consumer groups
  • Cassandra with partition_key=user_id and clustering_key=timestamp for feed metadata storage enabling efficient range queries
  • Encrypted object storage (AWS S3) for large feed content blobs to reduce DB load and enable zero-trust storage
  • Client-side encryption with user-specific keys to ensure end-to-end encryption and zero-trust backend
  • Cursor-based pagination with opaque tokens for efficient feed fetching and offline delivery
  • API design with idempotent POST /feed/post, paginated GET /feed, and delivery ACK POST /feed/ack endpoints
  • Autoscaling Kafka and Redis clusters to handle 10x viral traffic spikes without latency degradation

Practice

(1/5)
1. A system needs to serve AI code completions with sub-100ms latency while ensuring up-to-date code context. Which component is best suited to provide fast, low-latency retrieval of relevant code snippets?
easy
A. A relational database with complex SQL queries for code indexing
B. An in-memory vector search engine with approximate nearest neighbor (ANN) indexing
C. A distributed file system storing raw code files for batch processing
D. A message queue to buffer code snippet requests before processing

Solution

  1. Step 1: Identify low-latency retrieval need

    Relational DBs (A) have high query latency for vector similarity; distributed file systems (C) are for storage, not fast retrieval; message queues (D) buffer requests but don't serve retrieval.

  2. Step 2: Recognize vector search role

    ANN-based in-memory vector search engines (B) provide fast approximate nearest neighbor lookups, ideal for sub-100ms latency code snippet retrieval.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    ANN vector search enables fast similarity queries [check]
Hint: Vector search engines excel at fast similarity retrieval [check]
Common Mistakes:
2. For a mobile app with limited control over HTTP headers and caching proxies, which API versioning strategy is best to ensure backward compatibility and cache friendliness?
medium
A. Embed version in the URL path (e.g., /v1/resource) to leverage caching and simple routing
B. Use query parameters to specify version, allowing flexible client control
C. Use a custom HTTP header like X-API-Version to specify version
D. Use content negotiation with Accept header media types for versioning

Solution

  1. Step 1: Consider mobile client and caching constraints

    Mobile clients and caching proxies often ignore or strip custom headers, making header-based versioning unreliable.

  2. Step 2: Evaluate URL versioning benefits

    Embedding version in URL ensures cache keys differ per version, enabling backward compatibility and cache friendliness.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    URL versioning works best with caching proxies and limited header control [check]
Hint: URL versioning beats headers for caching and mobile clients [check]
Common Mistakes:
3. If the gRPC server's message queue is removed, what exactly breaks in the message delivery flow?
medium
A. Clients receive duplicate messages due to lack of deduplication
B. The server blocks sending until the client acknowledges, causing backpressure
C. Messages are lost if the client is slow or disconnected, with no retry mechanism
D. The server switches to synchronous unary calls, increasing latency

Solution

  1. Step 1: Role of message queue in gRPC server

    The queue buffers messages for slow or disconnected clients, enabling retries and flow control.

  2. Step 2: Effect of removing the queue

    Without the queue, messages sent to slow/disconnected clients are dropped immediately, causing message loss without retries.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    No queue -> message loss on slow/disconnected clients [check]
Hint: No queue -> messages lost if client slow or disconnected [check]
Common Mistakes:
4. You must implement a distributed lock for a system with high contention and strict latency requirements. Which approach is best and why?
medium
A. Use Redis SETNX with a fixed TTL to minimize latency
B. Use a consensus-based lock service like etcd to ensure correctness despite latency
C. Use a database row lock with retries to handle contention
D. Use a local in-memory lock combined with periodic heartbeat checks

Solution

  1. Step 1: Evaluate Redis SETNX under high contention

    Redis SETNX with TTL (A) is fast but can cause split-brain and stale locks under network partitions, violating correctness.

  2. Step 2: Consider consensus-based lock service

    Consensus-based services like etcd (B) provide strong consistency and fault tolerance, ensuring correctness even if latency is higher.

  3. Step 3: Assess database row locks and local locks

    Database locks (C) have high latency and risk deadlocks; local locks (D) do not coordinate across nodes, failing correctness.

  4. Final Answer:

    Option B -> Option B

  5. Quick Check:

    Consensus ensures correctness under contention despite latency [check]
Hint: Consensus beats speed for correctness under contention [check]
Common Mistakes:
5. At 10x traffic scale, which component in a multi-CDN architecture is most likely to become the first bottleneck, and why?
hard
A. Origin server, because it must serve all cache misses and cannot be horizontally scaled easily
B. Client DNS resolvers, because they cache CDN IPs and cannot update fast enough to reflect CDN health changes
C. Edge caches, because they have limited storage and cannot handle increased request volume
D. Global Traffic Manager, because it maintains stateful health checks and routing decisions for millions of users

Solution

  1. Step 1: Identify scaling challenges of Global Traffic Manager

    It maintains real-time health and routing state for all CDNs and users, which grows with traffic scale.

  2. Step 2: Compare other components

    Origin servers (A) can be scaled horizontally with caching. Edge caches (C) scale by adding nodes. Client DNS (D) caching delays but does not bottleneck traffic flow.

  3. Step 3: Conclude bottleneck

    Global Traffic Manager's centralized stateful routing logic is hardest to shard and scale linearly.

  4. Final Answer:

    Option D -> Option D

  5. Quick Check:

    Global Traffic Manager stateful routing -> first bottleneck at 10x scale [check]
Hint: Global Traffic Manager stateful routing limits scale first [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard