UMAP for dimensionality reduction in ML Python - ML Experiment: Train & Evaluate

Experiment - UMAP for dimensionality reduction

Problem:You have a dataset with many features, making it hard to visualize or analyze. You want to reduce the data to 2 dimensions while keeping the important structure.

Current Metrics:No dimensionality reduction applied yet; visualization is cluttered and unclear.

Issue:High dimensional data is difficult to visualize and understand. Need to reduce dimensions without losing important information.

Your Task

Use UMAP to reduce the dataset from high dimensions to 2 dimensions and visualize the result clearly.

Use UMAP only for dimensionality reduction.

Keep the original dataset unchanged.

Visualize the 2D output with a scatter plot colored by class labels.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
import umap

# Load example dataset
iris = load_iris()
X = iris.data
y = iris.target

# Create UMAP reducer
reducer = umap.UMAP(n_neighbors=15, min_dist=0.1, random_state=42)

# Fit and transform data
X_umap = reducer.fit_transform(X)

# Plot the 2D UMAP result
plt.figure(figsize=(8,6))
scatter = plt.scatter(X_umap[:, 0], X_umap[:, 1], c=y, cmap='viridis', s=50)
plt.title('UMAP projection of Iris dataset')
plt.xlabel('UMAP Dimension 1')
plt.ylabel('UMAP Dimension 2')
plt.colorbar(scatter, label='Class label')
plt.grid(True)
plt.show()

Imported umap library and used UMAP class for dimensionality reduction.

Set n_neighbors=15 and min_dist=0.1 to balance local and global structure.

Transformed original data to 2D using UMAP.

Visualized the 2D output with a scatter plot colored by class labels.

Results Interpretation

Before UMAP: Data in 4D space, hard to visualize or interpret.

After UMAP: Data projected to 2D with clear clusters matching class labels, easier to analyze and visualize.

UMAP effectively reduces high-dimensional data to low dimensions while preserving meaningful structure, helping us visualize and understand complex data.

Bonus Experiment

Try changing the n_neighbors and min_dist parameters to see how the UMAP visualization changes.

💡 Hint

Lower n_neighbors focuses on local structure, higher values capture more global structure. min_dist controls how tightly points are packed.

Practice

1. What is the main purpose of using UMAP in machine learning?

easy

A. To reduce the number of features while keeping data structure

B. To increase the number of features for better accuracy

C. To split data into training and testing sets

D. To normalize data values between 0 and 1

Solution

1. Step 1: Understand UMAP's role

   UMAP is a tool to reduce many features into fewer dimensions.

2. Step 2: Identify the goal of dimensionality reduction

   The goal is to keep similar data points close and preserve structure while reducing features.

3. Final Answer:

   To reduce the number of features while keeping data structure -> Option A

4. Quick Check:

   UMAP reduces features = B [OK]

Hint: UMAP shrinks features, keeps data shape [OK]

Common Mistakes:

• Thinking UMAP increases features
• Confusing UMAP with data splitting
• Mixing UMAP with normalization

2. Which of the following is the correct way to import UMAP from the umap-learn library in Python?

easy

A. from umap import umap

B. from umap import UMAP

C. import UMAP from umap

D. import umap.UMAP

Solution

1. Step 1: Recall correct Python import syntax

   Python imports classes or functions using 'from module import Class'.

2. Step 2: Match with UMAP library usage

   The correct import is 'from umap import UMAP'. Options A and C look similar but A uses lowercase 'umap' which is incorrect.

3. Final Answer:

   from umap import UMAP -> Option B

4. Quick Check:

   Correct import syntax = D [OK]

Hint: Use 'from umap import UMAP' to import [OK]

Common Mistakes:

• Using incorrect import syntax
• Confusing module and class names
• Using lowercase instead of uppercase for UMAP

3. What will be the shape of the output after applying UMAP with n_components=2 on a dataset with 100 samples and 50 features?

medium

A. (2, 50)

B. (50, 2)

C. (100, 2)

D. (100, 50)

Solution

1. Step 1: Understand input data shape

   The dataset has 100 samples (rows) and 50 features (columns).

2. Step 2: Apply UMAP dimensionality reduction

   UMAP reduces features from 50 to 2, so output shape is (samples, new_features) = (100, 2).

3. Final Answer:

   (100, 2) -> Option C

4. Quick Check:

   Output shape = (samples, n_components) = (100, 2) [OK]

Hint: Output rows = samples, columns = n_components [OK]

Common Mistakes:

• Swapping samples and features in output shape
• Confusing n_components with number of samples
• Assuming output shape stays same as input

4. You run UMAP with n_neighbors=5 on a dataset but get an error. What is the most likely cause?

medium

A. UMAP requires n_neighbors to be exactly 10

B. The dataset has more than 5 features

C. n_neighbors must be larger than number of features

D. The dataset has fewer than 5 samples

Solution

1. Step 1: Understand n_neighbors parameter

   n_neighbors defines how many nearest points UMAP uses to learn structure.

2. Step 2: Check dataset size relation

   If dataset has fewer samples than n_neighbors, UMAP cannot find enough neighbors, causing error.

3. Final Answer:

   The dataset has fewer than 5 samples -> Option D

4. Quick Check:

   n_neighbors ≤ samples needed = A [OK]

Hint: n_neighbors must be ≤ number of samples [OK]

Common Mistakes:

• Confusing features with samples for n_neighbors
• Assuming fixed n_neighbors value required
• Ignoring dataset size when setting n_neighbors

5. You want to visualize a dataset with 1000 samples and 100 features in 3D using UMAP. Which combination of parameters is best?

hard

A. n_components=3, n_neighbors=15 to balance detail and speed

B. n_components=2, n_neighbors=50 for maximum neighbor info

C. n_components=3, n_neighbors=1000 to use all samples as neighbors

D. n_components=10, n_neighbors=5 for detailed high dimensions

Solution

1. Step 1: Choose n_components for 3D visualization

   Set n_components=3 to get 3D output suitable for plotting.

2. Step 2: Select n_neighbors for balance

   n_neighbors=15 is a good default to capture local structure without slowing down too much.

3. Step 3: Evaluate other options

   n_components=2, n_neighbors=50 for maximum neighbor info uses 2D, not 3D. n_components=3, n_neighbors=1000 to use all samples as neighbors uses too many neighbors, slowing computation. n_components=10, n_neighbors=5 for detailed high dimensions uses 10 components, not 3D.

4. Final Answer:

   n_components=3, n_neighbors=15 to balance detail and speed -> Option A

5. Quick Check:

   3D + balanced neighbors = C [OK]

Hint: Use n_components=3 for 3D, moderate n_neighbors for speed [OK]

Common Mistakes:

• Choosing wrong n_components for visualization
• Setting n_neighbors too high causing slow run
• Confusing number of neighbors with number of components