Mean shift clustering in ML Python - ML Experiment: Train & Evaluate

Experiment - Mean shift clustering

Problem:You want to group data points into clusters without knowing the number of clusters beforehand. You use mean shift clustering on a 2D dataset.

Current Metrics:The model groups data but creates too many small clusters, making it hard to interpret. Cluster count is 15, but expected is around 3-5.

Issue:The bandwidth parameter is too small, causing over-segmentation and many tiny clusters.

Your Task

Adjust the mean shift clustering bandwidth to reduce the number of clusters to between 3 and 5, improving cluster quality and interpretability.

You can only change the bandwidth parameter.

Do not change the dataset or the clustering algorithm.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import MeanShift, estimate_bandwidth

# Generate sample data
np.random.seed(42)
cluster_centers = [[1, 1], [5, 5], [9, 1]]
data = []
for center in cluster_centers:
    data.append(np.random.randn(50, 2) + center)
data = np.vstack(data)

# Estimate bandwidth
bandwidth = estimate_bandwidth(data, quantile=0.3)

# Apply Mean Shift with adjusted bandwidth
ms = MeanShift(bandwidth=bandwidth)
ms.fit(data)
labels = ms.labels_
cluster_centers = ms.cluster_centers_

# Plot results
plt.figure(figsize=(8, 6))
colors = ['r', 'g', 'b', 'y', 'c', 'm']
for k in range(len(cluster_centers)):
    cluster_data = data[labels == k]
    plt.scatter(cluster_data[:, 0], cluster_data[:, 1], c=colors[k % len(colors)], label=f'Cluster {k}')
plt.scatter(cluster_centers[:, 0], cluster_centers[:, 1], c='k', marker='x', s=100, label='Centers')
plt.title(f'Mean Shift Clustering with bandwidth={bandwidth:.2f}')
plt.legend()
plt.show()

# Print number of clusters
print(f'Number of clusters: {len(cluster_centers)}')

Used sklearn's estimate_bandwidth with quantile=0.3 to find a better bandwidth value.

Set the MeanShift bandwidth parameter to the estimated value instead of default.

This merges small clusters into larger meaningful clusters.

Results Interpretation

Before: 15 clusters, many small groups, hard to interpret.

After: 3 clusters, matching expected groups, clearer cluster centers.

Adjusting the bandwidth in mean shift clustering controls cluster size. A larger bandwidth merges close points into fewer clusters, reducing over-segmentation.

Bonus Experiment

Try using a smaller bandwidth than the estimated one and observe how the number of clusters changes.

💡 Hint

Decrease the quantile parameter in estimate_bandwidth to get a smaller bandwidth and see if clusters split more.

Practice

1. What is the main idea behind mean shift clustering?

easy

A. It moves points toward areas with many nearby points to find clusters.

B. It assigns points randomly to clusters without considering neighbors.

C. It requires the number of clusters to be fixed before running.

D. It uses a decision tree to split data into clusters.

Solution

1. Step 1: Understand mean shift clustering concept

   Mean shift clustering works by shifting points toward the densest area nearby, grouping points naturally.

2. Step 2: Compare options with concept

   Only It moves points toward areas with many nearby points to find clusters. describes moving points toward dense areas. Others describe unrelated methods.

3. Final Answer:

   It moves points toward areas with many nearby points to find clusters. -> Option A

4. Quick Check:

   Mean shift = moves points to dense areas [OK]

Hint: Mean shift moves points to dense spots, no fixed cluster count [OK]

Common Mistakes:

• Thinking mean shift needs fixed cluster count
• Confusing mean shift with random assignment
• Believing mean shift uses decision trees

2. Which of the following is the correct way to import MeanShift from scikit-learn in Python?

easy

A. import MeanShift from sklearn.cluster

B. from sklearn.cluster import MeanShift

C. from sklearn import MeanShift

D. import sklearn.cluster.MeanShift

Solution

1. Step 1: Recall correct import syntax in Python

   Python uses 'from module import class' to import specific classes.

2. Step 2: Match syntax to options

   from sklearn.cluster import MeanShift uses 'from sklearn.cluster import MeanShift', which is correct. Others have wrong syntax.

3. Final Answer:

   from sklearn.cluster import MeanShift -> Option B

4. Quick Check:

   Correct import = from module import class [OK]

Hint: Use 'from module import class' to import specific classes [OK]

Common Mistakes:

• Using 'import' with 'from' incorrectly
• Trying to import class directly from package
• Missing 'import' keyword or wrong order

3. What will be the output cluster centers after running this code?

from sklearn.cluster import MeanShift
import numpy as np
X = np.array([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])
ms = MeanShift(bandwidth=2)
ms.fit(X)
print(ms.cluster_centers_)

medium

A. [[1. 2.] [10. 2.]]

B. [[1. 2.] [10. 4.]]

C. [[1. 2.] [10. 0.]]

D. [[5.5 2. ] [10. 2.]]

Solution

1. Step 1: Understand bandwidth and data points

   Bandwidth=2 means points within distance 2 form clusters. Points near (1,2) cluster together; points near (10,2) cluster together.

2. Step 2: Identify cluster centers

   Points at (1,0), (1,2), (1,4) average to (1,2). Points at (10,0), (10,2), (10,4) average to (10,2).

3. Final Answer:

   [[1. 2.] [10. 2.]] -> Option A

4. Quick Check:

   Clusters center near mean of close points [OK]

Hint: Clusters center near average of close points within bandwidth [OK]

Common Mistakes:

• Confusing cluster centers with original points
• Ignoring bandwidth effect on grouping
• Averaging points incorrectly

4. Identify the error in this MeanShift clustering code:

from sklearn.cluster import MeanShift
X = [[1, 2], [2, 3], [3, 4]]
ms = MeanShift()
ms.fit(X)
print(mss.labels_)

medium

A. Variable name 'ms' is used before assignment.

B. Input data X should be a NumPy array, not a list.

C. MeanShift requires bandwidth parameter to be set explicitly.

D. The print statement uses 'mss' but the object is named 'ms'.

Solution

1. Step 1: Check variable assignments and usage

   The clustering object is assigned to variable ms.

2. Step 2: Examine the print statement

   The print statement attempts to access mss.labels_, but mss is undefined. This will raise a NameError.

3. Step 3: Match to options

   The print statement uses 'mss' but the object is named 'ms'. correctly describes this issue: the print uses 'mss' while the object is 'ms'.

4. Final Answer:

   The print statement uses 'mss' but the object is named 'ms'. -> Option D

5. Quick Check:

   Typo in variable name causes runtime error [OK]

Hint: Check variable names carefully for typos in print statements [OK]

Common Mistakes:

• Assuming bandwidth is always required
• Thinking lists are invalid input
• Confusing variable names in print

5. You have a dataset with two dense groups close together and some scattered points far away. How should you set the bandwidth parameter in MeanShift to correctly identify the two main clusters?

hard

A. Set bandwidth to zero to get exact points as clusters.

B. Set bandwidth larger than the distance between the two groups to merge them.

C. Set bandwidth smaller than the distance between the two groups to separate them.

D. Set bandwidth equal to zero to ignore scattered points.

Solution

1. Step 1: Understand bandwidth effect on clustering

   Bandwidth controls neighborhood size. Smaller bandwidth means clusters form from closer points only.

2. Step 2: Apply to two close groups

   To keep two groups separate, bandwidth must be smaller than distance between groups, so they don't merge.

3. Step 3: Consider scattered points

   Scattered points may form their own clusters or be ignored depending on bandwidth, but main goal is separating main groups.

4. Final Answer:

   Set bandwidth smaller than the distance between the two groups to separate them. -> Option C

5. Quick Check:

   Bandwidth < distance = separate clusters [OK]

Hint: Bandwidth smaller than group distance keeps clusters separate [OK]

Common Mistakes:

• Setting bandwidth too large merges clusters
• Using zero bandwidth causes errors
• Ignoring scattered points effect