Bird
Raised Fist0

News Feed System - Personalization at Scale

SYSTEM-DESIGNhardInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🤖
News Feed System - Personalization at Scale
Design a scalable personalized news feed system
system-designhardFreq: very_highRecommendations
Prerequisites:Feed Ranking SystemRecommendation System
🎯
Problem Framing
Business Objective

Maximize user engagement by increasing time spent on the platform and interaction with content.

ML Objective

Maximize P(click|user, item, context) to personalize the news feed for each user.

Constraints
  • P99 latency < 100ms for feed generation
  • Real-time data processing for user interactions
  • Cold start problem for new users and items
  • Compliance with data privacy regulations (e.g., GDPR)
  • Scalability to millions of users and items
Clarifying Questions
  • What is the expected scale in terms of daily active users?
  • Are there any specific content types that need prioritization?
  • How frequently does the feed need to be updated?
  • What are the key engagement metrics the business cares about?
  • Are there any existing systems or data sources we need to integrate with?
Out of Scope
✗ Content moderation and filtering
✗ Backend infrastructure scaling
✗ User interface design
📊
Offline Metrics
PR-AUC: Preferred due to class imbalance with low click-through rates.
NDCG@10: Focuses on ranking quality by emphasizing top positions in the feed.
📈
Online Metrics
CTR: Directly measures user interaction with the feed.
Dwell Time: Measures the time users spend engaging with the feed.
⚖️
Trade-offs

Offline metrics like PR-AUC may not always correlate with online metrics like CTR due to real-world factors such as user interface changes or external events.

💼
Business Metric

Average session duration per user, as it reflects overall engagement and platform stickiness.

🗄️
Data Sources
  • User interaction logs (clicks, likes, shares) - real-time
  • Content metadata (tags, categories) - updated daily
  • User profiles (demographics, preferences) - updated weekly
  • Historical engagement data - batch processed
🏷️
Data Properties
Labeling Strategy

Implicit feedback from user interactions (clicks, likes) as it naturally reflects user preferences.

Data Volume

Approximately 500 million rows/day, 10 TB/month

Freshness

Features should be updated in real-time to reflect the latest user interactions.

⚠️
Data Challenges
  • Sparse labels due to low click-through rates
  • Position bias affecting click data
  • Survivorship bias in historical data
  • Cold start for new users and items
🔧
Key Features
embedding
User Embedding
Captures user preferences and behavior patterns.
embedding
Content Embedding
Represents content characteristics and similarities.
categorical
Time of Day
Accounts for temporal patterns in user activity.
categorical
Device Type
Different devices may influence user interaction.
numerical
Historical CTR
Provides context on past user engagement.
⚡
Feature Engineering Notes
Real-time vs Batch

User interaction features need real-time computation, while content embeddings can be pre-computed.

Feature Store

Required

🐍
Feature Snippet
Python
import numpy as np

def compute_user_embedding(user_id, interaction_history):
    # Example: Average embeddings of interacted items
    return np.mean([get_item_embedding(item) for item in interaction_history], axis=0)
🤖
Model Selection
Baseline

Popularity-based model using item click counts.

Proposed Model

Two-tower neural network model

The two-tower model efficiently captures user-item interactions and allows pre-computation of item embeddings, reducing online latency.
✗
Alternatives Rejected
Matrix Factorization: Rejected due to inability to handle new users/items without retraining.
XGBoost: Rejected due to high latency and memory overhead for real-time predictions.
🏋️
Training Strategy

Hybrid approach with offline training for base models and online learning for adapting to new data.

🚀
Serving Infrastructure
LatencyP99 < 100ms end-to-end
Throughput10,000 QPS
ArchitectureTwo-stage (retrieval + ranking) to efficiently handle large candidate sets.
CachingCache top N recommendations per user with a TTL of 5 minutes to reduce computation load.
FallbackUse a rule-based system based on content popularity if the model is unavailable.
📡
Data Drift Signals
Shift in user interaction distribution
Change in content popularity trends
Anomalies in feature distributions
📉
Model Drift Signals
Decrease in PR-AUC over time
Shift in prediction score distribution
Increase in prediction latency
🔔
Alerting & Retraining
Retraining Trigger

PR-AUC drops by more than 5% over a week.

Alerting Strategy

On-call engineer paged for significant metric deviations; logs monitored for latency spikes; auto-recovery for minor prediction failures.

🎯
Interview Angles
  • How to handle cold start problems effectively?
  • What strategies ensure low latency at high scale?
  • How to balance between offline and online metrics?
⚠️
Common Mistakes
Ignoring the impact of position bias in click data.
Overlooking the need for real-time feature updates.
Choosing complex models without considering latency constraints.
Failing to address data privacy and compliance issues.
📌
Key Decisions
  • Choosing a two-tower model for efficient embedding handling.
  • Implementing a hybrid training strategy for adaptability.
  • Designing a robust caching and fallback mechanism.

Practice

(1/5)
1. Which metric is most appropriate for evaluating the performance of a Two-Tower Model for Retrieval?
easy
A. Mean Squared Error (MSE)
B. Log Loss
C. F1 Score
D. Precision at K

Solution

  1. Step 1: Identify the goal of the retrieval model

    The goal is to rank items effectively for users, making ranking metrics like Precision at K suitable.

  2. Step 2: Eliminate wrong options

    Mean Squared Error (MSE) (MSE) is for regression tasks. F1 Score (F1 Score) is for classification tasks. Log Loss (Log Loss) is also for classification tasks.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Use ranking metrics like Precision at K. [OK]
Hint: Use ranking metrics like Precision at K. [OK]
Common Mistakes:
2. A hate speech detection model is suddenly showing a high false negative rate. What is the most likely cause?
medium
A. The training data is imbalanced with more negative samples.
B. The model's learning rate is too high.
C. The model is overfitting the training data.
D. The feature extraction process is incorrect.

Solution

  1. Step 1: Analyze the symptom

    A high false negative rate often indicates that the model is not sensitive enough to positive samples.

  2. Step 2: Eliminate wrong options

    Overfitting would increase false positives, a high learning rate causes instability, and incorrect feature extraction would affect all predictions.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    High false negatives = Imbalanced data [OK]
Hint: High false negatives = Imbalanced data [OK]
Common Mistakes:
3. In an LLM serving system, what is a likely failure mode if the KV Cache is not properly managed?
medium
A. Inaccurate predictions due to speculative decoding
B. Decreased throughput due to batching inefficiencies
C. Higher memory usage leading to system crashes
D. Increased latency due to redundant computations

Solution

  1. Step 1: Identify the role of KV Cache.

    KV Cache prevents redundant computations, reducing latency.

  2. Step 2: Eliminate wrong options.

    Batching inefficiencies affect throughput, not KV Cache. Memory issues are possible but less direct. Speculative decoding affects accuracy, not KV Cache.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    KV Cache mismanagement leads to redundant computations. [OK]
Hint: KV Cache mismanagement leads to redundant computations. [OK]
Common Mistakes:
4. A search ranking system suddenly shows irrelevant results at the top. What is the most likely failure mode?
medium
A. Query-document feature mismatch
B. Data leakage from training to test set
C. Overfitting due to too many features
D. Index corruption

Solution

  1. Step 1: Analyze the symptom

    Irrelevant results suggest a mismatch in understanding query-document relevance.

  2. Step 2: Identify the most likely cause

    Feature mismatch can lead to poor relevance scoring.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Feature mismatch leads to poor relevance. [OK]
Hint: Feature mismatch leads to poor relevance. [OK]
Common Mistakes:
5. As the user base of an ML platform grows 10x, which component is most likely to become a bottleneck first?
hard
A. Model Training Pipeline
B. Feature Store
C. Serving Infrastructure
D. Data Ingestion Pipeline

Solution

  1. Step 1: Consider the impact of scaling on each component.

    Serving Infrastructure is directly impacted by increased user requests, making it a likely bottleneck.

  2. Step 2: Eliminate wrong options.

    Model Training and Data Ingestion are less affected by user growth, and Feature Store scales with data, not users.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Serving scales with user requests. [OK]
Hint: Serving scales with user requests. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard