Bird
Raised Fist0

People You May Know - Graph-Based Link Prediction

SYSTEM-DESIGNhardInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🎯
Design a Graph-Based 'People You May Know' System
45 min · senior
senior

Your team of 5 engineers has 8 months to build a 'People You May Know' feature for a social network with 10M DAU. The system must handle 500 QPS with a P99 latency SLA of 200ms. You have access to user interaction logs and a social graph but no explicit friendship labels.

Given Info
  • The system must scale to 10M daily active users.
  • P99 latency SLA is 200ms for recommendations.
  • You have user interaction logs and a social graph.
  • No explicit friendship labels are available.
  • The team consists of 5 engineers with 8 months to deliver.
📋
Guided Steps
1/5
Step 1
Data Understanding and Preprocessing
How will you preprocess the available data to make it suitable for a graph-based link prediction model?
💡 Hint
Consider the types of features that can be derived from interaction logs and the social graph.
Key Decision
Feature engineering from interaction logs and graph structure because these provide implicit signals of user connections.
Interviewer Expectation
Mid: Identifies basic features like common neighbors. Senior: Proposes advanced features like personalized PageRank and node embeddings.
⚠️Common Wrong Answer
Ignoring the graph structure and relying solely on interaction logs, leading to poor prediction quality.
Step 2
Model Selection
What model architecture will you choose for link prediction, and why?
💡 Hint
Think about the trade-off between model complexity and serving latency.
Key Decision
Graph Neural Networks (GNNs) because they can capture complex relationships in graph data while maintaining reasonable latency.
Interviewer Expectation
Mid: Suggests a basic GNN model. Senior: Justifies the choice with scalability and latency considerations, and compares with simpler models like logistic regression on graph features.
⚠️Common Wrong Answer
Choosing a complex model like a deep GNN without considering the latency impact, leading to SLA violations.
Step 3
System Architecture
Design the overall system architecture for serving link predictions in real-time.
💡 Hint
Consider how to balance real-time inference with batch processing for model updates.
Key Decision
Hybrid architecture with real-time inference and periodic batch updates because it balances latency and model freshness.
Interviewer Expectation
Mid: Describes a basic real-time serving pipeline. Senior: Includes batch processing for model retraining and explains the trade-offs.
⚠️Common Wrong Answer
Designing a purely batch-based system, which fails to meet real-time latency requirements.
Step 4
Evaluation Metrics
What metrics will you use to evaluate the model both offline and online?
💡 Hint
Differentiate between metrics that measure model accuracy and those that reflect business impact.
Key Decision
Use PR-AUC for offline evaluation and CTR for online evaluation because they capture model performance and user engagement respectively.
Interviewer Expectation
Mid: Mentions basic metrics like accuracy. Senior: Explains the importance of PR-AUC for imbalanced data and CTR for business impact.
⚠️Common Wrong Answer
Focusing only on offline metrics like accuracy, which do not reflect real-world performance.
Step 5
Monitoring and Maintenance
How will you monitor the system in production to ensure it meets performance and reliability standards?
💡 Hint
Consider both model performance and system health metrics.
Key Decision
Implement monitoring for drift detection and latency spikes because these are critical for maintaining model accuracy and meeting SLAs.
Interviewer Expectation
Mid: Sets up basic monitoring for latency. Senior: Includes drift detection and explains how to respond to detected issues.
⚠️Common Wrong Answer
Neglecting to monitor for data drift, leading to degraded model performance over time.
📊
Scoring Rubric
▼ Show
Must Have - 60%
  • Proposes a graph-based model like GNNs and justifies it with scalability and latency considerations.
  • Distinguishes offline metric (PR-AUC for imbalanced data) from online metric (CTR for engagement) and explains why they can disagree.
  • Designs a hybrid system architecture that balances real-time inference with batch updates.
  • Includes a monitoring strategy for both model performance (drift detection) and system health (latency monitoring).
Good to Have - 30%
  • Suggests advanced graph features like personalized PageRank or node embeddings.
  • Explains the trade-offs between model complexity and serving latency.
  • Includes a plan for periodic model retraining to handle data drift.
  • Proposes a fallback mechanism for when the model fails to make predictions.
Bonus - 10%
  • Quantifies the expected latency impact of the chosen model architecture.
  • Discusses the potential for using transfer learning to improve model performance.
  • Explains how to handle cold start problems for new users.
✅
Model Answer
▼ Reveal

The ideal solution involves using a Graph Neural Network (GNN) for link prediction, leveraging both interaction logs and the social graph for feature engineering. The system should be designed as a hybrid architecture with real-time inference and periodic batch updates to balance latency and model freshness. Monitoring should include both performance metrics like PR-AUC and CTR, and system health metrics like latency and drift detection.

Key Decisions
  • Choosing GNNs for their ability to capture complex graph relationships while maintaining reasonable latency.
  • Implementing a hybrid architecture to balance real-time inference with batch updates for model retraining.
  • Using PR-AUC for offline evaluation and CTR for online evaluation to capture both model accuracy and business impact.
  • Setting up monitoring for data drift and latency spikes to ensure ongoing model performance and reliability.
Anti-patterns
Selecting a model without considering the latency impact, leading to SLA violations.
Ignoring the need for online metrics, resulting in a system that does not align with business goals.
Failing to plan for scalability, causing the system to struggle under the required load.

Practice

(1/5)
1. For an ad click-through rate prediction model, which metric is most appropriate to optimize for business value?
easy
A. Recall
B. Precision
C. AUC-ROC
D. Accuracy

Solution

  1. Step 1: Understand business objectives.

    CTR prediction models need to balance true positive rate and false positive rate.

  2. Step 2: Eliminate wrong options.

    Accuracy (A) doesn't account for class imbalance. Precision (B) and Recall (C) focus on one aspect of performance, not the trade-off.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    AUC-ROC balances trade-offs in CTR. [OK]
Hint: AUC-ROC balances trade-offs in CTR. [OK]
Common Mistakes:
2. In the context of a fine-tuning pipeline using SFT, RLHF, and DPO, which component is primarily responsible for aligning the model's outputs with human preferences?
easy
A. Reinforcement Learning from Human Feedback (RLHF)
B. Supervised Fine-Tuning (SFT)
C. Data Preprocessing Optimization (DPO)
D. Hyperparameter Tuning

Solution

  1. Step 1: Identify the role of each component

    SFT is used for initial model training, RLHF aligns outputs with human preferences, DPO optimizes data preprocessing, and hyperparameter tuning adjusts model parameters.

  2. Step 2: Eliminate wrong options

    A: SFT is for initial training, not alignment. C: DPO is unrelated to output alignment. D: Hyperparameter tuning is for model performance, not alignment.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    RLHF aligns with human feedback. [OK]
Hint: RLHF aligns with human feedback. [OK]
Common Mistakes:
3. In an ad auction system, when should you prefer a simpler linear model over a complex neural network for ranking ads?
medium
A. When interpretability and quick updates are crucial.
B. When the system needs to handle non-linear interactions.
C. When the dataset is extremely large and diverse.
D. When the computational resources are abundant.

Solution

  1. Step 1: Identify the scenario requirements

    In scenarios where interpretability and quick updates are crucial, a simpler linear model is preferred due to its transparency and ease of updating.

  2. Step 2: Eliminate wrong options

    • Extremely large and diverse dataset: Complex models handle this better.
    • Non-linear interactions: Neural networks are better suited.
    • Abundant computational resources: Allows for complex models.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Choose linear for interpretability and updates. [OK]
Hint: Choose linear for interpretability and updates. [OK]
Common Mistakes:
4. In a two-stage retrieval system, when should you prioritize a simpler candidate generation model over a complex one?
medium
A. When the ranking stage is computationally expensive and needs more resources.
B. When precision is more important than recall in the candidate generation stage.
C. When the dataset is small and computation resources are abundant.
D. When the ranking model is not capable of handling large candidate sets.

Solution

  1. Step 1: Consider resource allocation

    If the ranking stage is computationally expensive, a simpler candidate generation model can save resources for the ranking stage.

  2. Step 2: Eliminate wrong options

    When the dataset is small and computation resources are abundant. is incorrect because a small dataset doesn't necessitate a simpler model. When precision is more important than recall in the candidate generation stage. is incorrect because recall is typically prioritized in candidate generation. When the ranking model is not capable of handling large candidate sets. is incorrect because the ranking model should handle the candidate set size.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Save resources for ranking if it's costly. [OK]
Hint: Save resources for ranking if it's costly. [OK]
Common Mistakes:
5. As the data volume increases tenfold in an ML monitoring platform, which component is most likely to become a bottleneck first?
hard
A. Retraining Scheduler
B. Alerting System
C. Drift Detection Module
D. Model Evaluation Component

Solution

  1. Step 1: Analyze component scalability

    The Drift Detection Module processes incoming data to identify changes, making it sensitive to data volume increases.

  2. Step 2: Consider computational load

    Drift detection often involves statistical tests or complex calculations, which can become a bottleneck with large data volumes.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Data increase = Drift bottleneck [OK]
Hint: Data increase = Drift bottleneck [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard