Bird
Raised Fist0

Feed Ranking System - Twitter, Instagram, Facebook at Scale

SYSTEM-DESIGNhardInterview freq: very_high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🎯
Design a Scalable Feed Ranking System for a Social Media Platform
45 min · senior
senior

Your team of 4 engineers has 6 months to implement a personalized feed ranking system for a social media platform with 10M DAU. You have access to implicit feedback (clicks, likes, shares) but no explicit ratings. The P99 latency SLA for serving recommendations is 100ms. Design the ML system.

Given Info
  • 10M daily active users
  • Implicit feedback available: clicks, likes, shares
  • No explicit ratings available
  • P99 latency SLA is 100ms
  • Team size: 4 engineers
📋
Guided Steps
1/5
Step 1
Problem Framing and Objective Setting
What are the business and ML objectives for this feed ranking system?
💡 Hint
Consider how user engagement translates into business value and what ML metrics align with this.
Key Decision
Define business objective as maximizing user engagement and ML objective as optimizing CTR and dwell time.
Interviewer Expectation
Mid: Identifies CTR as a key metric. Senior: Also considers dwell time and explains how these metrics align with business goals.
⚠️Common Wrong Answer
Focusing solely on maximizing AUC without considering real-time engagement metrics.
Step 2
Data Pipeline and Feature Engineering
How will you design the data pipeline and feature engineering process?
💡 Hint
Think about data freshness and the types of features that can be derived from implicit feedback.
Key Decision
Use a real-time data pipeline with feature stores to ensure data freshness and support online learning.
Interviewer Expectation
Mid: Proposes batch processing for feature extraction. Senior: Suggests a hybrid approach with real-time updates for critical features.
⚠️Common Wrong Answer
Relying solely on batch processing, leading to stale features and reduced recommendation quality.
Step 3
Model Selection and Training
What model architecture will you choose and why?
💡 Hint
Consider the trade-offs between model complexity, training time, and inference latency.
Key Decision
Select a two-tower model for scalability and efficient inference, rejecting complex models like deep neural networks due to latency constraints.
Interviewer Expectation
Mid: Chooses a simple model like logistic regression. Senior: Justifies a two-tower model for its balance of performance and latency.
⚠️Common Wrong Answer
Choosing a complex model like a deep neural network without considering inference latency constraints.
Step 4
Serving and Scalability
How will you ensure the system meets the P99 latency requirement?
💡 Hint
Consider the impact of model size and feature retrieval on latency.
Key Decision
Deploy models on a distributed serving infrastructure with caching to minimize latency.
Interviewer Expectation
Mid: Proposes a single server setup. Senior: Designs a distributed system with caching and load balancing.
⚠️Common Wrong Answer
Underestimating the impact of network latency and not using caching effectively.
Step 5
Monitoring and Iteration
What monitoring strategies will you implement to ensure system reliability and performance?
💡 Hint
Think about both model performance and system health metrics.
Key Decision
Implement monitoring for CTR, latency, and feature drift to ensure system reliability.
Interviewer Expectation
Mid: Monitors basic system metrics like latency. Senior: Also tracks feature drift and user engagement metrics.
⚠️Common Wrong Answer
Ignoring feature drift, leading to degraded model performance over time.
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Defines business and ML objectives clearly, aligning them with engagement metrics.
  • Designs a data pipeline that ensures data freshness and supports real-time updates.
  • Selects a model architecture that balances performance with latency constraints.
  • Proposes a serving infrastructure that meets the P99 latency requirement.
  • Implements a comprehensive monitoring strategy including feature drift detection.
Good to Have - 30%
  • Considers user privacy and data security in the design.
  • Explains trade-offs between model complexity and inference latency.
  • Designs for scalability to handle future growth in user base.
  • Incorporates user feedback loops for continuous improvement.
Bonus - 10%
  • Proposes a hybrid model approach combining collaborative filtering with content-based methods.
  • Suggests advanced feature engineering techniques like embeddings for user and item profiles.
  • Considers A/B testing strategies for model evaluation and iteration.
✅
Model Answer
▼ Reveal

The ideal solution involves designing a scalable feed ranking system that maximizes user engagement through personalized recommendations. The system should leverage a real-time data pipeline to ensure data freshness and use a two-tower model for efficient inference. A distributed serving infrastructure with caching will meet the P99 latency requirement, and comprehensive monitoring will ensure system reliability.

Key Decisions
  • Define business objective as maximizing user engagement and ML objective as optimizing CTR and dwell time.
  • Use a real-time data pipeline with feature stores to ensure data freshness and support online learning.
  • Select a two-tower model for scalability and efficient inference, rejecting complex models like deep neural networks due to latency constraints.
  • Deploy models on a distributed serving infrastructure with caching to minimize latency.
  • Implement monitoring for CTR, latency, and feature drift to ensure system reliability.
Anti-patterns
Focusing solely on offline metrics without considering real-time engagement.
Neglecting data freshness, leading to outdated recommendations.
Choosing overly complex models that cannot meet latency requirements.
Ignoring system scalability and future growth potential.

Practice

(1/5)
1. For a model registry focused on governance, which metric is most critical to track?
easy
A. Model accuracy
B. Model compliance with regulations
C. Model deployment frequency
D. Model training time

Solution

  1. Step 1: Identify governance focus

    Governance in model registries is about ensuring compliance and adherence to regulations.

  2. Step 2: Eliminate wrong options

    Model accuracy is incorrect as accuracy isn't a governance metric. Model deployment frequency is wrong because deployment frequency isn't about governance. Model training time is incorrect as training time isn't related to governance.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Governance = Compliance focus [OK]
Hint: Governance = Compliance focus [OK]
Common Mistakes:
2. When choosing between pairwise and listwise approaches for a ranking task, which scenario favors the listwise approach?
medium
A. When the ranking task requires optimizing the entire list's order
B. When the ranking task involves a large number of items
C. When computational resources are limited
D. When the goal is to optimize for individual item relevance

Solution

  1. Step 1: Identify the scenario where listwise is beneficial.

    Listwise approaches are best when optimizing the entire list's order is crucial.

  2. Step 2: Eliminate wrong options.

    Pairwise is better for individual relevance, and listwise can be resource-intensive, making A and B incorrect.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Listwise = full list optimization [OK]
Hint: Listwise = full list optimization [OK]
Common Mistakes:
3. In an ML platform, when should you prefer batch processing over real-time processing for feature computation?
medium
A. When computational resources are limited and latency is not critical.
B. When the system requires immediate user feedback.
C. When features need to be updated every second.
D. When the model needs to adapt instantly to new data.

Solution

  1. Step 1: Analyze the constraints.

    Batch processing is suitable when resources are limited and latency is not a priority.

  2. Step 2: Eliminate wrong options.

    Options A, B, and D require real-time processing due to their need for immediacy.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Batch for non-critical latency. [OK]
Hint: Batch processing suits non-critical latency scenarios. [OK]
Common Mistakes:
4. What is a common failure mode when lineage tracking is not implemented in a model registry?
medium
A. Inability to reproduce model results
B. Increased model training time
C. Higher model deployment costs
D. Decreased model accuracy

Solution

  1. Step 1: Understand lineage tracking purpose

    Lineage tracking ensures that all steps and data leading to a model's creation are documented, aiding reproducibility.

  2. Step 2: Eliminate wrong options

    Increased model training time is incorrect as lineage doesn't affect training time. Higher model deployment costs is wrong because deployment costs aren't directly related to lineage. Decreased model accuracy is incorrect as accuracy isn't directly impacted by lineage tracking.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Lineage = Reproducibility [OK]
Hint: Lineage = Reproducibility [OK]
Common Mistakes:
5. When deploying a Two-Tower Model for Retrieval, which approach is better for handling cold-start users and why?
medium
A. Use random embeddings until enough data is collected.
B. Use collaborative filtering to generate initial embeddings.
C. Use content-based features to generate initial embeddings.
D. Use embeddings from a pre-trained language model.

Solution

  1. Step 1: Consider the cold-start problem

    Cold-start users lack interaction data, making content-based features useful for initial embeddings.

  2. Step 2: Eliminate wrong options

    Use collaborative filtering to generate initial embeddings. relies on interaction data, which cold-start users lack. Use random embeddings until enough data is collected. provides no meaningful information. Use embeddings from a pre-trained language model. might not capture user-specific features effectively.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Content-based features help with cold-start. [OK]
Hint: Content-based features help with cold-start. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard