Bird
Raised Fist0

Feed Ranking System - Twitter, Instagram, Facebook at Scale

SYSTEM-DESIGNhardInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🤖
Feed Ranking System - Twitter, Instagram, Facebook at Scale
Design a scalable feed ranking system for social media platforms.
system-designhardFreq: very_highRecommendations
Prerequisites:Learning to RankTwo-Stage Retrieval
🎯
Problem Framing
Business Objective

Maximize user engagement by increasing the time spent on the platform.

ML Objective

Maximize P(engagement|user, post, context) to rank posts in a user's feed.

Constraints
  • P99 latency < 100ms for ranking
  • Real-time data processing
  • Handling cold start for new users and posts
  • Compliance with privacy regulations (e.g., GDPR)
Clarifying Questions
  • What is the target latency for feed ranking?
  • How frequently does the feed need to be updated?
  • What is the scale of daily active users?
  • Are there any specific privacy constraints to consider?
  • How do we handle new users with no interaction history?
Out of Scope
✗ Ad ranking
✗ Content moderation
✗ User authentication
📊
Offline Metrics
NDCG@10 because it weights top results more, aligning with user engagement goals
PR-AUC due to class imbalance in engagement data
📈
Online Metrics
CTR (Click-Through Rate)
Average session duration
⚖️
Trade-offs

Offline metrics like NDCG@10 may not always correlate with online metrics like CTR due to differences in user behavior in live environments.

💼
Business Metric

Total time spent on platform per user

🗄️
Data Sources
  • User interaction logs (clicks, likes, shares) - real-time
  • Post metadata (author, timestamp, tags) - updated hourly
  • User profile data (demographics, interests) - updated daily
  • Social graph data (follows, connections) - updated daily
🏷️
Data Properties
Labeling Strategy

Implicit feedback from user interactions such as clicks and likes.

Data Volume

Approximately 1 billion interactions per day, 10 TB/month

Freshness

Features should be updated in real-time to reflect the latest user interactions.

⚠️
Data Challenges
  • Sparse labels due to low engagement rates
  • Position bias where earlier posts in the feed get more attention
  • Survivorship bias from only considering active users
  • Cold start for new users and posts
🔧
Key Features
feat
feat
feat
feat
feat
⚡
Feature Engineering Notes
Real-time vs Batch

User embeddings and interaction history need real-time updates, while post embeddings can be pre-computed.

Feature Store

Required

🐍
Feature Snippet
Python
def compute_user_embedding(user_id):
    # Fetch user interaction history
    interactions = get_user_interactions(user_id)
    # Compute embedding
    embedding = model.encode(interactions)
    return embedding
🤖
Model Selection
Baseline

Popularity-based ranking

Proposed Model

Two-tower neural network model

The two-tower model allows for efficient pre-computation of item embeddings and captures user-item interactions effectively, meeting the latency constraints.
✗
Alternatives Rejected
Logistic Regression - fails to capture complex user-item interactions
XGBoost - adds latency and memory overhead incompatible with real-time constraints
Collaborative Filtering - struggles with cold start and lacks real-time adaptability
🏋️
Training Strategy

Hybrid approach with offline training for base model and online learning for real-time updates.

🚀
Serving Infrastructure
LatencyP99 < 100ms end-to-end
Throughput10,000 QPS
ArchitectureTwo-stage (retrieval + ranking) to efficiently handle large candidate sets and refine top results.
CachingCache top N results per user with a TTL of 5 minutes to reduce computation load.
FallbackUse a rule-based system to rank posts based on recency and popularity if the model is unavailable.
📡
Data Drift Signals
Shift in user interaction distribution
Change in post metadata distribution
Alteration in social graph structure
📉
Model Drift Signals
Decrease in NDCG@10
Drop in CTR
Shift in prediction score distribution
🔔
Alerting & Retraining
Retraining Trigger

Retrain when NDCG@10 drops by more than 5% over a week.

Alerting Strategy

Page on-call engineer if CTR drops by more than 10% in a day; auto-recover by rolling back to the previous model version.

🎯
Interview Angles
  • How do you handle cold start problems in this system?
  • What are the trade-offs between offline and online metrics?
  • How do you ensure compliance with privacy regulations in data handling?
⚠️
Common Mistakes
Ignoring latency constraints in model selection
Overfitting to offline metrics without considering online performance
Neglecting real-time data freshness requirements
Failing to address cold start issues adequately
📌
Key Decisions
  • Choosing a two-tower model for efficient real-time inference
  • Implementing a hybrid training strategy for adaptability
  • Designing a robust caching strategy to meet latency requirements

Practice

(1/5)
1. For a visual search system, which metric is most appropriate to evaluate the quality of the image embeddings?
easy
A. F1 Score
B. Mean Squared Error
C. Accuracy
D. Precision at K

Solution

  1. Step 1: Understand the evaluation context

    Precision at K measures how many of the top K retrieved items are relevant, which is crucial for search systems.

  2. Step 2: Eliminate wrong options

    Mean Squared Error is for regression, Accuracy and F1 Score are for classification, not retrieval quality.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Search quality = Precision at K. [OK]
Hint: Search quality = Precision at K. [OK]
Common Mistakes:
2. In a scenario where model accuracy is critical, which approach should be avoided to prevent potential accuracy degradation?
medium
A. KV Cache
B. Speculative Decoding
C. Batching
D. Model Pruning

Solution

  1. Step 1: Understand the impact on accuracy.

    Speculative Decoding can introduce errors by predicting future tokens.

  2. Step 2: Eliminate wrong options.

    Batching and KV Cache do not inherently affect accuracy. Model Pruning reduces model size but can be controlled to maintain accuracy.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Speculative Decoding risks accuracy for speed. [OK]
Hint: Speculative Decoding risks accuracy for speed. [OK]
Common Mistakes:
3. What is a common failure mode when lineage tracking is not implemented in a model registry?
medium
A. Inability to reproduce model results
B. Increased model training time
C. Higher model deployment costs
D. Decreased model accuracy

Solution

  1. Step 1: Understand lineage tracking purpose

    Lineage tracking ensures that all steps and data leading to a model's creation are documented, aiding reproducibility.

  2. Step 2: Eliminate wrong options

    Increased model training time is incorrect as lineage doesn't affect training time. Higher model deployment costs is wrong because deployment costs aren't directly related to lineage. Decreased model accuracy is incorrect as accuracy isn't directly impacted by lineage tracking.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Lineage = Reproducibility [OK]
Hint: Lineage = Reproducibility [OK]
Common Mistakes:
4. When choosing between FAISS and HNSW for a vector search task, which is more suitable for handling dynamic datasets with frequent updates?
medium
A. HNSW, because it provides better precision.
B. FAISS, because it supports dynamic updates efficiently.
C. FAISS, because it has better indexing speed.
D. HNSW, because it allows for efficient updates and deletions.

Solution

  1. Step 1: Evaluate update capabilities.

    HNSW supports dynamic updates efficiently.

  2. Step 2: Eliminate wrong options.

    FAISS is not optimized for frequent updates; indexing speed and precision are not the primary concerns here.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    HNSW handles dynamic updates well. [OK]
Hint: HNSW handles dynamic updates well. [OK]
Common Mistakes:
5. At 10x data scale in a multimodal embedding pipeline, which component is most likely to become a bottleneck first?
hard
A. Model Inference
B. Feature Storage
C. Data Preprocessing
D. Network Bandwidth

Solution

  1. Step 1: Analyze the impact of scaling on each component.

    Feature Storage is likely to become a bottleneck due to increased data volume.

  2. Step 2: Eliminate wrong options.

    Data Preprocessing and Model Inference can be parallelized, and Network Bandwidth is less affected by data volume.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Storage struggles with data scale. [OK]
Hint: Storage struggles with data scale. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard