Bird
Raised Fist0

The Graph Disconnection

SYSTEM-DESIGNhardInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🐛
The Graph Disconnection
Started two weeks ago, ongoing.

A People You May Know feature using graph-based link prediction serving 10M users daily.

Symptom
AUC dropped from 0.85 to 0.70 over the past two weeks. · Recommendation diversity collapsed, with many users seeing repeated suggestions. · Real-time feature latency spiked from 50ms to 200ms.
📡
Signals
metric
AUC on live traffic
Dropped from 0.85 to 0.70
distribution
Distribution of recommended connections
Top 10 recommendations are repeated for 30% of users
latency
Real-time feature computation latency
Increased from 50ms to 200ms
feature_importance
Feature 'mutual_friends_count' importance
Dropped from 0.25 to 0.05 after last update
log
Error logs from feature computation service
Increased number of timeout errors
🔬
Hypotheses
Feature computation service is overloaded, causing latency spikes.
Supporting Signals
latencylog
Verify: Check server load and request queue lengths.
import psutil; print(psutil.cpu_percent(interval=1))
Recent data pipeline change introduced stale data.
Supporting Signals
metricdistribution
Verify: Compare timestamps of data used in training vs. live predictions.
df['timestamp'].describe()
🎯
Root Cause
A recent update to the data pipeline caused stale data to be used in real-time predictions.
Why It Happened

A misconfiguration in the data freshness check allowed outdated data to pass through.

🔧
Fix Strategy
ImmediateRollback to the last known good data pipeline configuration.
Short-termFix the data freshness check to ensure only current data is used.
Long-termImplement a robust data validation layer to catch stale data before it reaches production.
🐍
Python Fix
Python
def check_data_freshness(df):
    current_time = pd.Timestamp.now()
    freshness_threshold = pd.Timedelta('1 day')
    return df[df['timestamp'] > current_time - freshness_threshold]
✗
Common Wrong Diagnoses
▼ Show
Model overfitting due to recent changes.
Why candidates think this: AUC drop suggests model performance issues.
Why wrong: Validation AUC is stable, indicating the model itself is not overfitting.
Feature importance changes indicate a model issue.
Why candidates think this: Feature importance drop suggests model degradation.
Why wrong: The importance drop is due to stale data, not a model issue.
📊
Monitoring to Add
  • Real-time data freshness checks with alerts for stale data.
  • Latency monitoring with alerts for spikes above 100ms.
  • Diversity metrics for recommendations with alerts for drops.
📋
Process Change

Implement a mandatory review process for data pipeline changes.

🏗️
Architectural Change

Introduce a data versioning system to track and validate data freshness.

💡
Key Insight
Data freshness is critical in real-time systems; stale data can mimic model degradation.

Practice

(1/5)
1. In the context of a feed ranking system for social media platforms, which component is primarily responsible for determining the order of posts shown to a user?
easy
A. Ranking algorithm
B. Feature extraction module
C. Data ingestion pipeline
D. User interface

Solution

  1. Step 1: Identify the role of each component

    The ranking algorithm is responsible for determining the order of posts based on relevance and user preferences.

  2. Step 2: Eliminate wrong options

    Data ingestion pipeline collects data, feature extraction module processes features, and user interface displays the results but does not determine order.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Ranking algorithm orders posts. [OK]
Hint: Ranking algorithm orders posts. [OK]
Common Mistakes:
2. For a session-based recommendation system, which metric is most appropriate to evaluate the model's ability to predict the next item in a sequence?
easy
A. Precision@K
B. Mean Absolute Error (MAE)
C. Root Mean Square Error (RMSE)
D. F1 Score

Solution

  1. Step 1: Identify metrics for ranking tasks

    Precision@K is used to evaluate the relevance of the top-K recommended items.

  2. Step 2: Eliminate metrics for regression/classification

    MAE and RMSE are for regression; F1 Score is for classification.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Precision@K for ranking relevance. [OK]
Hint: Precision@K for ranking relevance. [OK]
Common Mistakes:
3. In a feed ranking system, should you prioritize using collaborative filtering or content-based filtering when user data is sparse?
medium
A. Collaborative filtering, because it leverages user interactions
B. Content-based filtering, because it relies on item features
C. Collaborative filtering, because it scales better with more users
D. Content-based filtering, because it requires less computational power

Solution

  1. Step 1: Analyze data sparsity

    When user data is sparse, collaborative filtering struggles due to lack of interaction data.

  2. Step 2: Eliminate wrong options

    Collaborative filtering needs user interactions. Content-based filtering uses item features, making it suitable for sparse data.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Content-based works with sparse data. [OK]
Hint: Content-based works with sparse data. [OK]
Common Mistakes:
4. As the number of concurrent experiments increases in an A/B testing platform, which component is most likely to become a bottleneck first?
hard
A. Data Storage Layer
B. Experiment Dashboard
C. Randomization Engine
D. User Segmentation Module

Solution

  1. Step 1: Consider the impact of scaling experiments.

    More experiments generate more data, stressing storage.

  2. Step 2: Eliminate wrong options.

    Dashboard and segmentation handle display and grouping, while randomization is not data-intensive.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    More experiments = more data. [OK]
Hint: More experiments = more data. [OK]
Common Mistakes:
5. As the number of users and ads increases tenfold in an ad auction system, which component is most likely to fail first?
hard
A. User Data Storage
B. Ad Ranking Module
C. Bid Management System
D. Network Bandwidth

Solution

  1. Step 1: Consider scale impact

    With a tenfold increase in users and ads, the User Data Storage is likely to face capacity issues first due to the sheer volume of data.

  2. Step 2: Eliminate wrong options

    • Ad Ranking Module: Computationally intensive but not storage-bound.
    • Bid Management System: Handles bids, not directly affected by user volume.
    • Network Bandwidth: Affects data transfer, but storage is a more immediate bottleneck.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Storage fails first with data volume increase. [OK]
Hint: Storage fails first with data volume increase. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard