Bird
Raised Fist0

Dimensionality Reduction - PCA, t-SNE, UMAP, When to Use Which

THEORYintermediateInterview freq: high
Choose your preparation mode6 modes available
LearnTraceDebugTrade-offChallenge
▶
Execution Trace
Step 1/4
PCA

PCA computes the covariance matrix and extracts eigenvectors, resulting in a linear transformation.

🎯 Understanding PCA's reliance on linear transformations.
from sklearn.decomposition import PCA
pca = PCA(n_components=2)
pca.fit(data)
t-SNE

t-SNE calculates pairwise similarities and optimizes the Kullback-Leibler divergence to preserve local structures.

🎯 Recognizing t-SNE's focus on preserving local structures.
from sklearn.manifold import TSNE
tsne = TSNE(n_components=2)
tsne.fit_transform(data)
UMAP

UMAP constructs a high-dimensional graph and optimizes a low-dimensional graph, efficiently preserving both local and global structures.

🎯 UMAP's efficiency in preserving both local and global structures.
import umap
umap_model = umap.UMAP(n_components=2)
umap_model.fit_transform(data)
Projection Type

PCA results in a linear projection, while t-SNE and UMAP provide non-linear projections.

🎯 Choosing the right technique based on data characteristics.
# Linear vs Non-linear decision
if data_is_linear:
    model = PCA(n_components=2)
else:
    model = TSNE(n_components=2)  # or UMAP
📊
Diagram
Diagram
┌───────────────────────────────┐
│       Dimensionality          │
│         Reduction             │
└───────────────────────────────┘
          │       │       │      
          ▼       ▼       ▼      
      ┌──────┐ ┌──────┐ ┌──────┐ 
      │  PCA │ │ t-SNE│ │ UMAP │ 
      └──────┘ └──────┘ └──────┘ 
          │       │       │      
          ▼       ▼       ▼      
  ┌────────────┐ ┌────────────┐  
  │ Linear     │ │ Non-linear │  
  │ Projection │ │ Projection │  
  └────────────┘ └────────────┘
Labels
  • Dimensionality Reduction: The process of reducing the number of random variables.
  • PCA: Principal Component Analysis, a linear technique.
  • t-SNE: t-Distributed Stochastic Neighbor Embedding, a non-linear technique.
  • UMAP: Uniform Manifold Approximation and Projection, a non-linear technique.
  • Linear/Non-linear Projection: The type of transformation applied to the data.
🔑
Key Moments
▲
Q: Why does PCA only capture linear relationships?
A: PCA relies on eigenvectors of the covariance matrix, which are linear transformations.
Q: How does t-SNE maintain local structure?
A: t-SNE minimizes the Kullback-Leibler divergence between pairwise similarities in high and low dimensions.
Q: What makes UMAP efficient for large datasets?
A: UMAP's graph-based approach efficiently captures both local and global data structures.
Trace Summary

This diagram and trace explain the mechanisms of three popular dimensionality reduction techniques: PCA, t-SNE, and UMAP. PCA is a linear method that projects data onto principal components, making it suitable for linearly separable data. t-SNE and UMAP, on the other hand, are non-linear techniques that excel at preserving local and global structures, respectively. The choice between these methods depends on the data's characteristics and the desired outcome, such as whether linearity or non-linearity is more important for the task at hand.

Primary Focus

✓ Dimensionality Reduction Techniques

Practice

(1/5)
1. In the context of Convolutional Neural Networks, what is the primary purpose of pooling layers?
easy
A. To reduce the spatial dimensions of the feature maps
B. To increase the size of the feature maps
C. To increase the number of channels in the feature maps
D. To apply non-linear activation functions

Solution

  1. Step 1: Understand pooling layers

    Pooling layers are used to reduce the spatial dimensions of feature maps, which helps in reducing computation and controlling overfitting.

  2. Step 2: Eliminate wrong options

    To increase the size of the feature maps is incorrect because pooling reduces size, not increases it. To increase the number of channels in the feature maps is incorrect as pooling doesn't change the number of channels. To apply non-linear activation functions is incorrect because pooling doesn't apply non-linear activations.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Pooling reduces spatial dimensions. [OK]
Hint: Pooling reduces spatial dimensions. [OK]
Common Mistakes:
2. 
import pandas as pd
from sklearn.impute import SimpleImputer

# Sample data
data = {'A': [1, 2, None, 4], 'B': [None, 2, 3, 4]}
df = pd.DataFrame(data)

# Imputation
imputer = SimpleImputer(strategy='mean')
imputed_data = imputer.fit_transform(df)
print(imputed_data)
What is the output of the above code?
easy
A. [[1.0, 3.0], [2.0, 2.0], [2.5, 3.0], [4.0, 4.0]]
B. [[1.0, 3.0], [2.0, 2.0], [2.333, 3.0], [4.0, 4.0]]
C. [[1.0, 3.0], [2.0, 2.0], [2.333, 3.0], [4.0, 4.0]]
D. [[1.0, 3.0], [2.0, 2.0], [2.5, 3.0], [4.0, 4.0]]

Solution

  1. Step 1: Calculate the mean for each column

    Column A mean: (1+2+4)/3 = 2.333, Column B mean: (2+3+4)/3 = 3.0

  2. Step 2: Apply mean imputation

    Replace missing values with calculated means.

  3. Final Answer:

    Option A -> Option A

  4. Quick Check:

    Mean replaces NaN [OK]
Hint: Mean replaces NaN [OK]
Common Mistakes:
3. A decision tree model is overfitting the training data. What is the most likely cause?
medium
A. The tree is too shallow.
B. The features are not informative.
C. The dataset is too small.
D. The tree is too deep.

Solution

  1. Step 1: Identify overfitting causes.

    Overfitting often occurs when the model is too complex, such as a deep tree.

  2. Step 2: Eliminate wrong options.

    A shallow tree would underfit, a small dataset affects generalization but not directly overfitting, and uninformative features would lead to poor performance, not overfitting.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Deep trees overfit easily. [OK]
Hint: Deep trees overfit easily. [OK]
Common Mistakes:
4. When choosing between polynomial features and interaction terms for a linear model, which is more appropriate when the dataset is small and why?
medium
A. Polynomial features, because they capture non-linear relationships better.
B. Interaction terms, because they increase model complexity less.
C. Polynomial features, because they require fewer parameters.
D. Interaction terms, because they are less prone to overfitting.

Solution

  1. Step 1: Consider dataset size and model complexity.

    Small datasets are prone to overfitting with complex models.

  2. Step 2: Evaluate the impact of each option.

    Polynomial features (A, C) increase complexity significantly, leading to overfitting. Interaction terms (B, D) add complexity but less than polynomial features, making them more suitable for small datasets.

  3. Final Answer:

    Option D -> Option D

  4. Quick Check:

    Small data: prefer interaction terms to avoid overfitting. [OK]
Hint: Small data: prefer interaction terms to avoid overfitting. [OK]
Common Mistakes:
5. A company implemented an ML model to predict customer churn but found that the model's predictions were not improving business outcomes. What is the most likely root cause?
medium
A. The model is overfitting the training data
B. The business problem was not correctly translated into an ML problem
C. The model's hyperparameters were not optimized
D. The dataset used was too small

Solution

  1. Step 1: Identify the mismatch

    If the model's predictions are not improving business outcomes, it is likely that the business problem was not correctly translated into an ML problem.

  2. Step 2: Eliminate wrong options

    The model is overfitting the training data is incorrect because overfitting would affect prediction accuracy, not business alignment. The model's hyperparameters were not optimized is incorrect because hyperparameter tuning affects model performance, not problem alignment. The dataset used was too small is incorrect because data size affects model accuracy, not problem framing.

  3. Final Answer:

    Option B -> Option B

  4. Quick Check:

    Align ML problem with business goals for impact. [OK]
Hint: Align ML problem with business goals for impact. [OK]
Common Mistakes:
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard