Bird
Raised Fist0

Dimensionality Reduction - PCA, t-SNE, UMAP, When to Use Which

THEORYintermediateInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
🐛
The Silent Recall Killer
Started after the last feature update two weeks ago.

A recommendation system for an e-commerce platform using dimensionality reduction to preprocess user-item interaction data.

Symptom
Model accuracy is stable, but recall has dropped to 0.
📡
Signals
metric
Recall metric on the test set
Recall dropped from 0.65 to 0.00 after the update
feature_importance
Importance of features after dimensionality reduction
Top features are now unrecognizable compared to previous runs
distribution
Distribution of reduced dimensions
New dimensions have a uniform distribution, unlike the previous Gaussian-like distribution
log
Logs from the dimensionality reduction step
t-SNE used instead of PCA in the latest run
code
Version control diff for preprocessing script
Change from PCA to t-SNE without parameter tuning
🔬
Hypotheses
The change from PCA to t-SNE caused the issue
Supporting Signals
logcode
Verify: Check if reverting to PCA restores recall
from sklearn.decomposition import PCA
pca = PCA(n_components=10)
X_reduced = pca.fit_transform(X)
# Evaluate model with X_reduced
t-SNE parameters are not optimized for the data
Supporting Signals
distributionfeature_importance
Verify: Experiment with different t-SNE parameters and check recall
from sklearn.manifold import TSNE
tsne = TSNE(n_components=10, perplexity=30)
X_reduced = tsne.fit_transform(X)
# Evaluate model with X_reduced
🎯
Root Cause
Switching from PCA to t-SNE without parameter tuning led to poor feature representation.
Why It Happened

A recent update changed the dimensionality reduction technique without proper validation.

🔧
Fix Strategy
ImmediateRollback to the previous version using PCA.
Short-termRe-evaluate the choice of dimensionality reduction technique and tune parameters.
Long-termImplement a validation step for any changes in preprocessing techniques.
🐍
Python Fix
Python
from sklearn.decomposition import PCA
pca = PCA(n_components=10)
X_reduced = pca.fit_transform(X)
# Re-train and evaluate model
✗
Common Wrong Diagnoses
▼ Show
Assuming data leakage due to unchanged validation AUC.
Why candidates think this: Stable validation AUC suggests no data leakage.
Why wrong: The issue was with feature representation, not data leakage.
Believing the model is overfitting due to unchanged accuracy.
Why candidates think this: Accuracy is stable, so overfitting seems plausible.
Why wrong: Recall dropping to zero indicates a feature representation issue, not overfitting.
📊
Monitoring to Add
  • Track recall and precision metrics continuously post-deployment.
  • Monitor distribution of reduced dimensions for anomalies.
  • Log changes in preprocessing techniques and parameters.
📋
Process Change

Introduce a mandatory review process for changes in data preprocessing steps.

🏗️
Architectural Change

Develop a modular preprocessing pipeline that allows easy experimentation and rollback.

💡
Key Insight
Switching dimensionality reduction techniques without proper validation can silently degrade model performance.

Practice

(1/5)
1. 
import numpy as np

# Simulate A/B test results
A_results = np.array([1, 0, 1, 1, 0])
B_results = np.array([0, 1, 0, 0, 1])

# Calculate mean conversion rates
A_mean = np.mean(A_results)
B_mean = np.mean(B_results)

# Determine which variant is better
better_variant = 'A' if A_mean > B_mean else 'B'
print(better_variant)
What does this code output?
easy
A. A
B. B
C. Error due to division by zero
D. Error due to shape mismatch

Solution

  1. Step 1: Calculate means

    A_mean = 0.6, B_mean = 0.2

  2. Step 2: Compare means

    A_mean > B_mean, so 'A' is printed.

  3. Final Answer:

    A -> A

  4. Quick Check:

    Calculate means, compare, print. [OK]
Hint: Calculate means, compare, print. [OK]
Common Mistakes:
2. 
import numpy as np
from sklearn.preprocessing import StandardScaler

X = np.array([[1, 2], [3, 4], [5, 6]])
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
print(X_scaled.mean(axis=0))
What does this code output?
easy
A. [1. 1.]
B. [0.5 0.5]
C. [0. 0.]
D. [0. 1.]

Solution

  1. Step 1: Understand standardization.

    StandardScaler standardizes features by removing the mean and scaling to unit variance.

  2. Step 2: Calculate the mean of standardized data.

    After standardization, the mean of each feature should be 0.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    StandardScaler centers data to mean 0. [OK]
Hint: StandardScaler centers data to mean 0. [OK]
Common Mistakes:
3. In the context of why machine learning exists, which of the following problems is most suitable for a machine learning approach rather than a rule-based system?
easy
A. Predicting the next word in a sentence
B. Calculating the sum of a list of numbers
C. Sorting a list of integers
D. Checking if a number is prime

Solution

  1. Step 1: Identify the nature of the problem.

    Predicting the next word in a sentence involves understanding context and patterns, which are complex and not easily defined by rules.

  2. Step 2: Eliminate rule-based problems.

    Options A, C, and D are deterministic and can be solved with explicit algorithms.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    ML for patterns, rules for logic. [OK]
Hint: ML excels at pattern recognition, not deterministic logic. [OK]
Common Mistakes:
4. In a scenario where computational resources are limited, which approach is more suitable for hyperparameter tuning and why?
medium
A. Use a train-validation split and skip the test set.
B. Use a single train-test split to save resources.
C. Use cross-validation to ensure robust hyperparameter tuning.
D. Use a larger test set to ensure model generalization.

Solution

  1. Step 1: Evaluate resource constraints

    Cross-validation is computationally expensive but provides robust tuning.

  2. Step 2: Eliminate less suitable options

    Use a single train-test split to save resources. risks overfitting. Use a train-validation split and skip the test set. skips the test set, risking unseen data performance. Use a larger test set to ensure model generalization. misallocates resources to testing rather than tuning.

  3. Final Answer:

    Option C -> Option C

  4. Quick Check:

    Cross-validation ensures robust tuning despite cost. [OK]
Hint: Cross-validation ensures robust tuning despite cost. [OK]
Common Mistakes:
5. You need to implement a real-time recommendation system. Which approach is most suitable given the need for low latency and high throughput?
hard
A. Online learning with a streaming framework
B. Offline training with periodic updates
C. Batch processing with Spark
D. Using a pre-trained model with batch inference

Solution

  1. Step 1: Identify the requirements.

    Real-time systems require low latency and high throughput.

  2. Step 2: Choose the approach that meets these requirements.

    Online learning with a streaming framework allows for real-time updates and low latency.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Streaming fits real-time needs. [OK]
Hint: Streaming fits real-time needs. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard