UMAP for dimensionality reduction in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - UMAP for dimensionality reduction

Which metric matters for UMAP and WHY

UMAP reduces data to fewer dimensions while keeping its shape. We check how well it keeps neighbors close. Trustworthiness and Continuity are key metrics. Trustworthiness shows if points close in low dimensions were close before. Continuity checks if original close points stay close after. These tell us if UMAP keeps the data's true structure.

Confusion matrix or equivalent visualization

UMAP does not classify, so no confusion matrix. Instead, we use neighbor preservation matrices. For example, a matrix showing how many original neighbors remain neighbors after reduction:

Original neighbors: 5
Neighbors after UMAP: 4
Preserved neighbors: 3
Trustworthiness = 3 / 4 = 0.75
Continuity = 3 / 5 = 0.6
    

This shows how many neighbors UMAP kept correctly.

Precision vs Recall tradeoff (or equivalent)

UMAP balances keeping local and global data shapes. Trustworthiness is like precision: it measures if neighbors in low dimensions are truly neighbors. Continuity is like recall: it checks if original neighbors appear in low dimensions. High trustworthiness but low continuity means UMAP shows only some neighbors well. High continuity but low trustworthiness means it shows many neighbors but some are wrong. We want both high for good reduction.

What "good" vs "bad" metric values look like

Good UMAP: Trustworthiness and continuity above 0.9 means neighbors are well kept. The low-dimensional map shows clear groups like original data.

Bad UMAP: Trustworthiness or continuity below 0.5 means many neighbors are lost or wrongly placed. The map looks mixed or confusing.

Common pitfalls with UMAP metrics

• Ignoring global structure: UMAP focuses on local neighbors, so global distances may distort.
• Overfitting neighbors: Too many neighbors in UMAP can force false connections, lowering trustworthiness.
• Using only visual checks: A pretty plot may hide poor neighbor preservation.
• Not comparing metrics: Trustworthiness or continuity alone can mislead; use both.

Self-check question

Your UMAP reduction has trustworthiness 0.95 but continuity 0.4. Is it good? Why or why not?

Answer: No, it is not good. High trustworthiness means neighbors shown are mostly correct, but low continuity means many original neighbors are missing. The map misses many true neighbors, so it does not fully keep the data's structure.

Key Result

UMAP quality is best judged by trustworthiness and continuity, measuring how well neighbors are preserved in reduced space.

Practice

1. What is the main purpose of using UMAP in machine learning?

easy

A. To reduce the number of features while keeping data structure

B. To increase the number of features for better accuracy

C. To split data into training and testing sets

D. To normalize data values between 0 and 1

Solution

1. Step 1: Understand UMAP's role

   UMAP is a tool to reduce many features into fewer dimensions.

2. Step 2: Identify the goal of dimensionality reduction

   The goal is to keep similar data points close and preserve structure while reducing features.

3. Final Answer:

   To reduce the number of features while keeping data structure -> Option A

4. Quick Check:

   UMAP reduces features = B [OK]

Hint: UMAP shrinks features, keeps data shape [OK]

Common Mistakes:

• Thinking UMAP increases features
• Confusing UMAP with data splitting
• Mixing UMAP with normalization

2. Which of the following is the correct way to import UMAP from the umap-learn library in Python?

easy

A. from umap import umap

B. from umap import UMAP

C. import UMAP from umap

D. import umap.UMAP

Solution

1. Step 1: Recall correct Python import syntax

   Python imports classes or functions using 'from module import Class'.

2. Step 2: Match with UMAP library usage

   The correct import is 'from umap import UMAP'. Options A and C look similar but A uses lowercase 'umap' which is incorrect.

3. Final Answer:

   from umap import UMAP -> Option B

4. Quick Check:

   Correct import syntax = D [OK]

Hint: Use 'from umap import UMAP' to import [OK]

Common Mistakes:

• Using incorrect import syntax
• Confusing module and class names
• Using lowercase instead of uppercase for UMAP

3. What will be the shape of the output after applying UMAP with n_components=2 on a dataset with 100 samples and 50 features?

medium

A. (2, 50)

B. (50, 2)

C. (100, 2)

D. (100, 50)

Solution

1. Step 1: Understand input data shape

   The dataset has 100 samples (rows) and 50 features (columns).

2. Step 2: Apply UMAP dimensionality reduction

   UMAP reduces features from 50 to 2, so output shape is (samples, new_features) = (100, 2).

3. Final Answer:

   (100, 2) -> Option C

4. Quick Check:

   Output shape = (samples, n_components) = (100, 2) [OK]

Hint: Output rows = samples, columns = n_components [OK]

Common Mistakes:

• Swapping samples and features in output shape
• Confusing n_components with number of samples
• Assuming output shape stays same as input

4. You run UMAP with n_neighbors=5 on a dataset but get an error. What is the most likely cause?

medium

A. UMAP requires n_neighbors to be exactly 10

B. The dataset has more than 5 features

C. n_neighbors must be larger than number of features

D. The dataset has fewer than 5 samples

Solution

1. Step 1: Understand n_neighbors parameter

   n_neighbors defines how many nearest points UMAP uses to learn structure.

2. Step 2: Check dataset size relation

   If dataset has fewer samples than n_neighbors, UMAP cannot find enough neighbors, causing error.

3. Final Answer:

   The dataset has fewer than 5 samples -> Option D

4. Quick Check:

   n_neighbors ≤ samples needed = A [OK]

Hint: n_neighbors must be ≤ number of samples [OK]

Common Mistakes:

• Confusing features with samples for n_neighbors
• Assuming fixed n_neighbors value required
• Ignoring dataset size when setting n_neighbors

5. You want to visualize a dataset with 1000 samples and 100 features in 3D using UMAP. Which combination of parameters is best?

hard

A. n_components=3, n_neighbors=15 to balance detail and speed

B. n_components=2, n_neighbors=50 for maximum neighbor info

C. n_components=3, n_neighbors=1000 to use all samples as neighbors

D. n_components=10, n_neighbors=5 for detailed high dimensions

Solution

1. Step 1: Choose n_components for 3D visualization

   Set n_components=3 to get 3D output suitable for plotting.

2. Step 2: Select n_neighbors for balance

   n_neighbors=15 is a good default to capture local structure without slowing down too much.

3. Step 3: Evaluate other options

   n_components=2, n_neighbors=50 for maximum neighbor info uses 2D, not 3D. n_components=3, n_neighbors=1000 to use all samples as neighbors uses too many neighbors, slowing computation. n_components=10, n_neighbors=5 for detailed high dimensions uses 10 components, not 3D.

4. Final Answer:

   n_components=3, n_neighbors=15 to balance detail and speed -> Option A

5. Quick Check:

   3D + balanced neighbors = C [OK]

Hint: Use n_components=3 for 3D, moderate n_neighbors for speed [OK]

Common Mistakes:

• Choosing wrong n_components for visualization
• Setting n_neighbors too high causing slow run
• Confusing number of neighbors with number of components