UMAP for dimensionality reduction in ML Python - Model Pipeline Trace

Model Pipeline - UMAP for dimensionality reduction

This pipeline uses UMAP to reduce the number of features in data while keeping its important structure. It helps us see and understand complex data by turning many features into just two or three.

Data Flow - 3 Stages

1Input Data

1000 rows x 50 columns→Raw data with 50 features per sample→1000 rows x 50 columns

[[0.5, 1.2, ..., 0.3], [1.1, 0.7, ..., 0.9], ...]

↓

2Preprocessing

1000 rows x 50 columns→Standardize features to zero mean and unit variance→1000 rows x 50 columns

[[-0.3, 0.8, ..., -1.1], [1.2, -0.5, ..., 0.4], ...]

↓

3UMAP Dimensionality Reduction

1000 rows x 50 columns→Reduce features from 50 to 2 using UMAP→1000 rows x 2 columns

[[1.5, -0.7], [0.3, 1.2], ...]

Training Trace - Epoch by Epoch

Loss
1.0 | *       
0.8 | **      
0.6 | ***     
0.4 | ****    
0.2 | *****   
    +---------
     1 2 3 4 5
     Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	N/A	Initial embedding with high loss, structure not clear
2	0.60	N/A	Loss decreased, clusters start to form
3	0.45	N/A	Better separation of groups visible
4	0.35	N/A	Embedding stabilizes, loss decreases slower
5	0.30	N/A	Final embedding with clear cluster structure

Prediction Trace - 3 Layers

Layer 1: Input Sample

Layer 2: Standardization

Layer 3: UMAP Projection

Model Quiz - 3 Questions

What does UMAP do in this pipeline?

A Increases the number of features

B Removes rows from the dataset

C Reduces data from many features to fewer features

D Changes data labels

Key Insight

UMAP helps us see complex data by turning many features into just a few while keeping the important patterns. Standardizing data first helps UMAP work better. The training loss going down shows the model finds a clearer view of the data step by step.

Practice

1. What is the main purpose of using UMAP in machine learning?

easy

A. To reduce the number of features while keeping data structure

B. To increase the number of features for better accuracy

C. To split data into training and testing sets

D. To normalize data values between 0 and 1

Solution

1. Step 1: Understand UMAP's role

   UMAP is a tool to reduce many features into fewer dimensions.

2. Step 2: Identify the goal of dimensionality reduction

   The goal is to keep similar data points close and preserve structure while reducing features.

3. Final Answer:

   To reduce the number of features while keeping data structure -> Option A

4. Quick Check:

   UMAP reduces features = B [OK]

Hint: UMAP shrinks features, keeps data shape [OK]

Common Mistakes:

• Thinking UMAP increases features
• Confusing UMAP with data splitting
• Mixing UMAP with normalization

2. Which of the following is the correct way to import UMAP from the umap-learn library in Python?

easy

A. from umap import umap

B. from umap import UMAP

C. import UMAP from umap

D. import umap.UMAP

Solution

1. Step 1: Recall correct Python import syntax

   Python imports classes or functions using 'from module import Class'.

2. Step 2: Match with UMAP library usage

   The correct import is 'from umap import UMAP'. Options A and C look similar but A uses lowercase 'umap' which is incorrect.

3. Final Answer:

   from umap import UMAP -> Option B

4. Quick Check:

   Correct import syntax = D [OK]

Hint: Use 'from umap import UMAP' to import [OK]

Common Mistakes:

• Using incorrect import syntax
• Confusing module and class names
• Using lowercase instead of uppercase for UMAP

3. What will be the shape of the output after applying UMAP with n_components=2 on a dataset with 100 samples and 50 features?

medium

A. (2, 50)

B. (50, 2)

C. (100, 2)

D. (100, 50)

Solution

1. Step 1: Understand input data shape

   The dataset has 100 samples (rows) and 50 features (columns).

2. Step 2: Apply UMAP dimensionality reduction

   UMAP reduces features from 50 to 2, so output shape is (samples, new_features) = (100, 2).

3. Final Answer:

   (100, 2) -> Option C

4. Quick Check:

   Output shape = (samples, n_components) = (100, 2) [OK]

Hint: Output rows = samples, columns = n_components [OK]

Common Mistakes:

• Swapping samples and features in output shape
• Confusing n_components with number of samples
• Assuming output shape stays same as input

4. You run UMAP with n_neighbors=5 on a dataset but get an error. What is the most likely cause?

medium

A. UMAP requires n_neighbors to be exactly 10

B. The dataset has more than 5 features

C. n_neighbors must be larger than number of features

D. The dataset has fewer than 5 samples

Solution

1. Step 1: Understand n_neighbors parameter

   n_neighbors defines how many nearest points UMAP uses to learn structure.

2. Step 2: Check dataset size relation

   If dataset has fewer samples than n_neighbors, UMAP cannot find enough neighbors, causing error.

3. Final Answer:

   The dataset has fewer than 5 samples -> Option D

4. Quick Check:

   n_neighbors ≤ samples needed = A [OK]

Hint: n_neighbors must be ≤ number of samples [OK]

Common Mistakes:

• Confusing features with samples for n_neighbors
• Assuming fixed n_neighbors value required
• Ignoring dataset size when setting n_neighbors

5. You want to visualize a dataset with 1000 samples and 100 features in 3D using UMAP. Which combination of parameters is best?

hard

A. n_components=3, n_neighbors=15 to balance detail and speed

B. n_components=2, n_neighbors=50 for maximum neighbor info

C. n_components=3, n_neighbors=1000 to use all samples as neighbors

D. n_components=10, n_neighbors=5 for detailed high dimensions

Solution

1. Step 1: Choose n_components for 3D visualization

   Set n_components=3 to get 3D output suitable for plotting.

2. Step 2: Select n_neighbors for balance

   n_neighbors=15 is a good default to capture local structure without slowing down too much.

3. Step 3: Evaluate other options

   n_components=2, n_neighbors=50 for maximum neighbor info uses 2D, not 3D. n_components=3, n_neighbors=1000 to use all samples as neighbors uses too many neighbors, slowing computation. n_components=10, n_neighbors=5 for detailed high dimensions uses 10 components, not 3D.

4. Final Answer:

   n_components=3, n_neighbors=15 to balance detail and speed -> Option A

5. Quick Check:

   3D + balanced neighbors = C [OK]

Hint: Use n_components=3 for 3D, moderate n_neighbors for speed [OK]

Common Mistakes:

• Choosing wrong n_components for visualization
• Setting n_neighbors too high causing slow run
• Confusing number of neighbors with number of components