Bird
Raised Fist0

Feed Ranking System - Twitter, Instagram, Facebook at Scale

SYSTEM-DESIGNhardInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
⚖️
Collaborative Filtering vs Deep Learning with Embeddings

For a social media feed ranking system with millions of users and items, would you use collaborative filtering or deep learning with embeddings? Justify your choice.

📊
Comparison
DimensionCollaborative FilteringDeep Learning with Embeddings
ScalabilityModerateHigh
Cold Start HandlingPoorGood
Training SpeedFastSlow
Inference LatencyLowModerate
ComplexityLowHigh
✅
Verdict
Default to collaborative filtering for moderate-sized datasets with dense interactions and limited compute resources. Switch to deep learning with embeddings when handling large-scale data with complex patterns and sufficient computational resources.
💡
Key Insight
The choice between collaborative filtering and deep learning hinges on the scale of data and available computational resources.
⚠️
Interview Trap
❌ Wrong: Always use deep learning because it handles large data better.
Ignoring the computational cost and complexity of deep learning when simpler methods suffice.
✅ Consider the trade-off between computational resources and the complexity of the problem. Use deep learning when the scale and complexity justify the cost.
Collaborative Filtering
"Excels at leveraging user-item interaction data without needing extensive feature engineering."
Weaknesses
  • Struggles with cold start problem for new users/items.
  • Limited to existing interactions, cannot generalize to unseen data.
  • Scalability issues with very large datasets due to matrix factorization complexity.
Ideal When
  • When user-item interaction data is dense (interaction matrix density > 0.1).
  • When the dataset size is moderate (n < 10M users/items).
  • When computational resources are limited (no GPU access).
Netflix uses collaborative filtering for its recommendation system to leverage user viewing history effectively.
Deep Learning with Embeddings
"Capable of capturing complex patterns and generalizing to new users/items through embeddings."
Weaknesses
  • Requires significant computational resources (GPU/TPU).
  • Longer training times compared to traditional methods.
  • Complexity in tuning and architecture design.
Ideal When
  • When dataset size is large (n > 100M users/items).
  • When you have access to high-performance computing resources (GPUs/TPUs).
  • When you need to handle cold start problems effectively.
Facebook uses deep learning with embeddings to handle the massive scale and complexity of user interactions on its platform.
🐍
Collaborative Filtering - Example
▼ Show
Python
from surprise import SVD, Dataset

# Load the movielens-100k dataset
data = Dataset.load_builtin('ml-100k')

# Use the SVD algorithm
algo = SVD()

# Train on the dataset
trainset = data.build_full_trainset()
algo.fit(trainset)
🐍
Deep Learning with Embeddings - Example
▼ Show
Python
import tensorflow as tf

# Define a simple embedding model
user_input = tf.keras.layers.Input(shape=(1,))
item_input = tf.keras.layers.Input(shape=(1,))

user_embedding = tf.keras.layers.Embedding(input_dim=10000, output_dim=64)(user_input)
item_embedding = tf.keras.layers.Embedding(input_dim=10000, output_dim=64)(item_input)

# Flatten and concatenate
user_vec = tf.keras.layers.Flatten()(user_embedding)
item_vec = tf.keras.layers.Flatten()(item_embedding)

concat = tf.keras.layers.Concatenate()([user_vec, item_vec])

# Dense layers
x = tf.keras.layers.Dense(128, activation='relu')(concat)
output = tf.keras.layers.Dense(1, activation='sigmoid')(x)

model = tf.keras.models.Model(inputs=[user_input, item_input], outputs=output)
model.compile(optimizer='adam', loss='binary_crossentropy')
✅
Use Collaborative Filtering When
  • •When interaction matrix density > 0.1.
  • •When n < 10M users/items.
  • •When computational resources are limited (no GPU access).
✅
Use Deep Learning with Embeddings When
  • •When n > 100M users/items.
  • •When you have access to GPUs/TPUs.
  • •When you need to handle cold start problems effectively.
↳
Nuances
  • Collaborative filtering can be surprisingly effective with dense interaction data even at moderate scales.
  • Deep learning models require careful tuning and architecture design to outperform simpler models.
  • Cold start problems can be mitigated with hybrid approaches combining collaborative filtering and content-based methods.
  • For very sparse datasets, deep learning with embeddings can leverage additional features to improve performance.

Practice

(1/5)
1. In an ad auction system, if the ads are not being displayed as expected, which failure mode is most likely?
medium
A. Data leakage from training to test set.
B. Latency in fetching user data.
C. Incorrect bid values being used.
D. Overfitting due to too many features.

Solution

  1. Step 1: Identify the symptom

    If ads are not displayed as expected, latency in fetching user data can cause delays, leading to missed ad opportunities.

  2. Step 2: Eliminate wrong options

    • Incorrect bid values: Affects ranking, not display.
    • Data leakage: Affects model accuracy, not display.
    • Overfitting: Affects model performance, not display timing.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Latency affects ad display timing. [OK]
Hint: Latency affects ad display timing. [OK]
Common Mistakes:
2. A hate speech detection model is suddenly showing a high false negative rate. What is the most likely cause?
medium
A. The training data is imbalanced with more negative samples.
B. The model's learning rate is too high.
C. The model is overfitting the training data.
D. The feature extraction process is incorrect.

Solution

  1. Step 1: Analyze the symptom

    A high false negative rate often indicates that the model is not sensitive enough to positive samples.

  2. Step 2: Eliminate wrong options

    Overfitting would increase false positives, a high learning rate causes instability, and incorrect feature extraction would affect all predictions.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    High false negatives = Imbalanced data [OK]
Hint: High false negatives = Imbalanced data [OK]
Common Mistakes:
3. In an LLM serving system, what is a likely failure mode if the KV Cache is not properly managed?
medium
A. Inaccurate predictions due to speculative decoding
B. Decreased throughput due to batching inefficiencies
C. Higher memory usage leading to system crashes
D. Increased latency due to redundant computations

Solution

  1. Step 1: Identify the role of KV Cache.

    KV Cache prevents redundant computations, reducing latency.

  2. Step 2: Eliminate wrong options.

    Batching inefficiencies affect throughput, not KV Cache. Memory issues are possible but less direct. Speculative decoding affects accuracy, not KV Cache.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    KV Cache mismanagement leads to redundant computations. [OK]
Hint: KV Cache mismanagement leads to redundant computations. [OK]
Common Mistakes:
4. In a search ranking system, when should you prefer a learning-to-rank model over a traditional BM25 algorithm?
medium
A. When you have a small dataset
B. When interpretability is crucial
C. When computational resources are limited
D. When you have rich user interaction data

Solution

  1. Step 1: Assess the data and resource context

    Learning-to-rank models excel with rich interaction data.

  2. Step 2: Evaluate the constraints

    BM25 is better for small datasets, interpretability, and limited resources.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Use learning-to-rank with rich interaction data. [OK]
Hint: Use learning-to-rank with rich interaction data. [OK]
Common Mistakes:
5. In a semantic search system using a Bi-Encoder, what is a likely failure mode if the document embeddings are not updated regularly?
medium
A. Increased computational cost during retrieval
B. Increased latency in query processing
C. Decreased retrieval accuracy due to outdated embeddings
D. Overfitting to the training data

Solution

  1. Step 1: Identify the impact of outdated embeddings

    Outdated embeddings can lead to decreased retrieval accuracy as they may not reflect current document content.

  2. Step 2: Eliminate wrong options

    Increased computational cost and latency are not directly related to outdated embeddings, and overfitting is a training issue.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Outdated embeddings = accuracy drop [check]
Hint: Outdated embeddings = accuracy drop [check]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard