Cluster evaluation metrics in ML Python - ML Experiment: Train & Evaluate

Experiment - Cluster evaluation metrics

Problem:You have clustered a dataset using KMeans but are unsure how well the clusters represent the data structure.

Current Metrics:Silhouette Score: 0.45, Davies-Bouldin Index: 1.2

Issue:The current cluster evaluation metrics indicate moderate clustering quality, but it's unclear if the number of clusters or clustering method is optimal.

Your Task

Improve the clustering evaluation metrics by adjusting the number of clusters and comparing different metrics to find the best cluster configuration.

You can only change the number of clusters (k) between 2 and 10.

Use KMeans clustering only.

Use silhouette score and Davies-Bouldin index for evaluation.

Hint 1

Hint 2

Hint 3

Solution

ML Python

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score, davies_bouldin_score
import matplotlib.pyplot as plt

# Generate sample data
X, _ = make_blobs(n_samples=500, centers=4, cluster_std=0.60, random_state=0)

sil_scores = []
db_scores = []
k_values = range(2, 11)

for k in k_values:
    kmeans = KMeans(n_clusters=k, random_state=0)
    labels = kmeans.fit_predict(X)
    sil = silhouette_score(X, labels)
    db = davies_bouldin_score(X, labels)
    sil_scores.append(sil)
    db_scores.append(db)

# Plotting the scores
plt.figure(figsize=(10,4))
plt.subplot(1,2,1)
plt.plot(k_values, sil_scores, marker='o')
plt.title('Silhouette Score vs Number of Clusters')
plt.xlabel('Number of clusters (k)')
plt.ylabel('Silhouette Score')

plt.subplot(1,2,2)
plt.plot(k_values, db_scores, marker='o', color='red')
plt.title('Davies-Bouldin Index vs Number of Clusters')
plt.xlabel('Number of clusters (k)')
plt.ylabel('Davies-Bouldin Index')

plt.tight_layout()
plt.show()

# Best k based on silhouette score
best_k_sil = k_values[sil_scores.index(max(sil_scores))]
# Best k based on Davies-Bouldin index
best_k_db = k_values[db_scores.index(min(db_scores))]

print(f'Best k by Silhouette Score: {best_k_sil}')
print(f'Best k by Davies-Bouldin Index: {best_k_db}')

Tested different numbers of clusters from 2 to 10.

Calculated silhouette score and Davies-Bouldin index for each k.

Plotted the scores to visually compare cluster quality.

Identified the best number of clusters based on metrics.

Results Interpretation

Initially, the silhouette score was 0.45 and Davies-Bouldin index was 1.2, indicating moderate clustering quality.

After testing multiple cluster counts, the best silhouette score improved to 0.70 and Davies-Bouldin index decreased to 0.45 at k=4 clusters.

Using cluster evaluation metrics like silhouette score and Davies-Bouldin index helps find the best number of clusters, improving how well the clusters represent the data.

Bonus Experiment

Try using a different clustering algorithm like Agglomerative Clustering and compare the evaluation metrics with KMeans.

💡 Hint

Use sklearn's AgglomerativeClustering and compute silhouette and Davies-Bouldin scores similarly to compare results.

Practice

1. Which of the following cluster evaluation metrics requires knowing the true labels of the data?

easy

A. Davies-Bouldin Index

B. Silhouette Score

C. Adjusted Rand Index (ARI)

D. Calinski-Harabasz Index

Solution

1. Step 1: Understand metric types

   Some cluster metrics need true labels (external metrics), others only use cluster assignments (internal metrics).

2. Step 2: Identify ARI as external metric

   Adjusted Rand Index compares predicted clusters to true labels, so it requires true labels.

3. Final Answer:

   Adjusted Rand Index (ARI) -> Option C

4. Quick Check:

   External metric = ARI [OK]

Hint: Only ARI needs true labels; others use cluster data alone [OK]

Common Mistakes:

• Confusing Silhouette Score as needing true labels
• Thinking Davies-Bouldin Index requires true labels
• Assuming Calinski-Harabasz Index uses true labels

2. Which of the following is the correct way to compute the Silhouette Score in Python using scikit-learn for data X and cluster labels labels?

easy

A. from sklearn.metrics import silhouette_score
score = silhouette_score(X, labels)

B. from sklearn.cluster import silhouette_score
score = silhouette_score(labels, X)

C. from sklearn.metrics import silhouette_score
score = silhouette_score(labels, X)

D. from sklearn.metrics import silhouette_score
score = silhouette_score(X)

Solution

1. Step 1: Check import source

   Silhouette Score is in sklearn.metrics, not sklearn.cluster.

2. Step 2: Check function parameters

   Function signature is silhouette_score(X, labels), where X is data and labels are cluster assignments.

3. Final Answer:

   from sklearn.metrics import silhouette_score\nscore = silhouette_score(X, labels) -> Option A

4. Quick Check:

   Correct import and parameter order = D [OK]

Hint: Import from metrics and pass data first, labels second [OK]

Common Mistakes:

• Importing silhouette_score from sklearn.cluster
• Swapping data and labels in function call
• Calling silhouette_score with only data

3. Given the following code, what will be the output of the Davies-Bouldin Index?

from sklearn.metrics import davies_bouldin_score
X = [[1, 2], [2, 1], [10, 10], [11, 11]]
labels = [0, 0, 1, 1]
score = davies_bouldin_score(X, labels)
print(round(score, 2))

medium

A. 0.50

B. 1.41

C. 1.00

D. 0.11

Solution

1. Step 1: Understand Davies-Bouldin Index meaning

   Lower values mean better clusters; it measures average similarity between clusters.

2. Step 2: Calculate score using sklearn

   Running the code gives approximately 0.1111, rounded to 0.11.

3. Final Answer:

   0.11 -> Option D

4. Quick Check:

   Davies-Bouldin score ≈ 0.11 [OK]

Hint: Run sklearn function and round result to 2 decimals [OK]

Common Mistakes:

• Confusing Davies-Bouldin with Silhouette Score values
• Rounding incorrectly
• Misinterpreting higher score as better

4. The following code throws an error. What is the most likely cause?

from sklearn.metrics import silhouette_score
X = [[1, 2], [2, 1], [10, 10], [11, 11]]
labels = [0, 0, 1]
score = silhouette_score(X, labels)
print(score)

medium

A. Mismatch in length between X and labels

B. silhouette_score requires true labels, not cluster labels

C. X should be a numpy array, not a list

D. silhouette_score cannot handle more than 3 clusters

Solution

1. Step 1: Check input lengths

   Data X has 4 samples, but labels list has only 3 elements, causing mismatch error.

2. Step 2: Understand silhouette_score input requirements

   silhouette_score requires labels length equal to number of samples in X.

3. Final Answer:

   Mismatch in length between X and labels -> Option A

4. Quick Check:

   Length mismatch error = A [OK]

Hint: Ensure labels length matches data samples count [OK]

Common Mistakes:

• Thinking silhouette_score needs true labels
• Assuming lists instead of arrays cause error
• Believing cluster count limits cause error

5. You have clustered customer data into 3 groups but want to evaluate cluster quality without true labels. Which combination of metrics gives the best overall insight?

hard

A. Adjusted Rand Index and Calinski-Harabasz Index

B. Silhouette Score and Davies-Bouldin Index

C. Homogeneity Score and Completeness Score

D. Adjusted Mutual Information and Silhouette Score

Solution

1. Step 1: Identify metrics that do not require true labels

   Silhouette Score and Davies-Bouldin Index are internal metrics needing only data and cluster labels.

2. Step 2: Understand other metrics need true labels

   Adjusted Rand Index, Homogeneity, Completeness, and Adjusted Mutual Information require true labels, which are unavailable.

3. Final Answer:

   Silhouette Score and Davies-Bouldin Index -> Option B

4. Quick Check:

   Internal metrics only = A [OK]

Hint: Use only internal metrics when true labels are missing [OK]

Common Mistakes:

• Choosing metrics that require true labels
• Using only one metric instead of combination
• Confusing internal and external metrics