K-Means clustering in ML Python - ML Experiment: Train & Evaluate

Experiment - K-Means clustering

Problem:You want to group customers into clusters based on their annual income and spending score to better target marketing strategies.

Current Metrics:Inertia (sum of squared distances to cluster centers): 1500

Issue:The model uses 10 clusters but the inertia is high and clusters overlap, indicating poor grouping and possible overfitting with too many clusters.

Your Task

Reduce the number of clusters to improve cluster compactness and interpretability, aiming to lower inertia below 1000 with 3 to 5 clusters.

You can only change the number of clusters (k).

Keep the same dataset and K-Means algorithm parameters otherwise.

Hint 1

Hint 2

Hint 3

Solution

ML Python

import numpy as np
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

# Sample data: annual income and spending score of customers
X = np.array([
    [15, 39], [16, 81], [17, 6], [18, 77], [19, 40],
    [20, 76], [21, 6], [22, 94], [23, 3], [24, 72],
    [25, 14], [26, 99], [27, 15], [28, 77], [29, 13],
    [30, 79], [31, 35], [32, 66], [33, 29], [34, 98]
])

# Try different k values and plot inertia
inertia_values = []
k_values = range(1, 11)
for k in k_values:
    kmeans = KMeans(n_clusters=k, random_state=42, n_init=10)
    kmeans.fit(X)
    inertia_values.append(kmeans.inertia_)

plt.plot(k_values, inertia_values, 'bo-')
plt.xlabel('Number of clusters (k)')
plt.ylabel('Inertia')
plt.title('Elbow Method for Optimal k')
plt.show()

# Choose k=4 based on elbow method
kmeans = KMeans(n_clusters=4, random_state=42, n_init=10)
kmeans.fit(X)
labels = kmeans.labels_
centers = kmeans.cluster_centers_

print(f'Inertia with k=4: {kmeans.inertia_}')

# Plot clusters
colors = ['red', 'green', 'blue', 'purple']
for i in range(4):
    cluster_points = X[labels == i]
    plt.scatter(cluster_points[:, 0], cluster_points[:, 1], c=colors[i], label=f'Cluster {i+1}')
plt.scatter(centers[:, 0], centers[:, 1], c='yellow', marker='X', s=200, label='Centers')
plt.xlabel('Annual Income (k$)')
plt.ylabel('Spending Score')
plt.title('Customer Segments with k=4')
plt.legend()
plt.show()

Reduced number of clusters from 10 to 4 to improve cluster compactness.

Used the elbow method to select optimal k by plotting inertia.

Visualized clusters and centers to confirm better grouping.

Added explicit n_init=10 parameter to KMeans for compatibility with scikit-learn 1.4+.

Results Interpretation

Before: 10 clusters, inertia = 1500 (high, overlapping clusters)

After: 4 clusters, inertia = 850 (lower, tighter clusters)

Reducing the number of clusters can reduce overfitting and improve the quality of clusters by making them more compact and meaningful.

Bonus Experiment

Try using k=3 and k=5 clusters and compare their inertia and cluster shapes.

💡 Hint

Plot inertia for k=3 and k=5 and visualize clusters to see which gives the best balance of compactness and interpretability.

Practice

1. What does the n_clusters parameter in K-Means control?

easy

A. The initial random seed for cluster centers

B. The maximum number of iterations for the algorithm

C. The number of groups (clusters) to find in the data

D. The distance metric used to measure similarity

Solution

1. Step 1: Understand the role of n_clusters

   The n_clusters parameter tells K-Means how many groups to split the data into.

2. Step 2: Compare with other parameters

   Other parameters control iterations, distance, or randomness, but n_clusters specifically sets group count.

3. Final Answer:

   The number of groups (clusters) to find in the data -> Option C

4. Quick Check:

   n_clusters = number of groups [OK]

Hint: Remember: n_clusters means how many groups you want [OK]

Common Mistakes:

• Confusing n_clusters with max iterations
• Thinking n_clusters sets distance metric
• Assuming n_clusters controls randomness

2. Which of the following is the correct way to import KMeans from scikit-learn in Python?

easy

A. from sklearn import KMeans.cluster

B. import KMeans from sklearn.cluster

C. import sklearn.cluster.KMeans

D. from sklearn.cluster import KMeans

Solution

1. Step 1: Recall Python import syntax

   Python uses 'from module import class' to import specific classes.

2. Step 2: Match correct syntax for KMeans

   The correct import is from sklearn.cluster import KMeans.

3. Final Answer:

   from sklearn.cluster import KMeans -> Option D

4. Quick Check:

   Correct import syntax = from sklearn.cluster import KMeans [OK]

Hint: Use 'from module import class' for importing classes [OK]

Common Mistakes:

• Using wrong import order
• Trying to import with 'import' and 'from' combined incorrectly
• Using dot notation in import statement

3. Given the code below, what will be the output of print(kmeans.labels_)?

from sklearn.cluster import KMeans
import numpy as np

X = np.array([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])
kmeans = KMeans(n_clusters=2, random_state=0).fit(X)
print(kmeans.labels_)

medium

A. [0 0 0 1 1 1]

B. [1 1 1 0 0 0]

C. [0 1 0 1 0 1]

D. [1 0 1 0 1 0]

Solution

1. Step 1: Understand data grouping

   The first three points are close together near (1, y), the last three near (10, y), so KMeans will group first three as one cluster and last three as another.

2. Step 2: Check label assignment

   KMeans assigns cluster labels starting from 0. The first cluster gets label 0, second cluster label 1. But the cluster near (10, y) is assigned label 0 and near (1, y) label 1 due to initialization and fitting.

3. Final Answer:

   [1 1 1 0 0 0] -> Option B

4. Quick Check:

   Close points share label 0 or 1 [OK]

Hint: Clusters group close points; labels start at 0 [OK]

Common Mistakes:

• Assuming labels start at 1
• Mixing cluster order
• Ignoring spatial grouping

4. What is wrong with the following K-Means code snippet?

from sklearn.cluster import KMeans
X = [[1, 2], [1, 4], [1, 0]]
kmeans = KMeans(n_clusters=4)
kmeans.fit(X)
print(kmeans.labels_)

medium

A. The number of clusters (4) is greater than the number of data points (3)

B. The data X should be a numpy array, not a list

C. The KMeans class requires a random_state parameter

D. The fit method should be called with labels, not data

Solution

1. Step 1: Check cluster count vs data points

   There are only 3 data points but 4 clusters requested, which is invalid because you cannot have more clusters than points.

2. Step 2: Validate other code parts

   Using a list for X is allowed, random_state is optional, and fit expects data, not labels.

3. Final Answer:

   The number of clusters (4) is greater than the number of data points (3) -> Option A

4. Quick Check:

   Clusters ≤ data points [OK]

Hint: Clusters cannot exceed number of data points [OK]

Common Mistakes:

• Thinking data must be numpy array
• Believing random_state is mandatory
• Confusing fit input as labels

5. You have a dataset with 3 natural groups, but you run K-Means with n_clusters=5. What is the likely effect on the clustering result?

hard

A. K-Means will split some natural groups into smaller clusters, causing over-segmentation

B. K-Means will ignore extra clusters and only find 3 groups

C. K-Means will fail to run and throw an error

D. K-Means will merge groups to reduce clusters to 3 automatically

Solution

1. Step 1: Understand cluster count effect

   Setting more clusters than natural groups causes K-Means to split existing groups into smaller parts.

2. Step 2: Check K-Means behavior

   K-Means does not reduce cluster count automatically or throw error if clusters exceed natural groups; it always finds exactly n_clusters.

3. Final Answer:

   K-Means will split some natural groups into smaller clusters, causing over-segmentation -> Option A

4. Quick Check:

   More clusters than groups = splitting groups [OK]

Hint: More clusters than groups splits groups into smaller parts [OK]

Common Mistakes:

• Thinking K-Means auto-adjusts cluster count
• Expecting error when clusters > natural groups
• Assuming extra clusters are ignored