Cluster evaluation metrics in SciPy - Step-by-Step Execution

Concept Flow - Cluster evaluation metrics

Start with true labels and predicted clusters

↓

Choose evaluation metric

↓

Calculate metric value

↓

Interpret metric: higher or lower is better

↓

Use metric to compare clustering quality

↓

End

We start with true and predicted cluster labels, pick a metric, calculate it, then interpret the result to judge clustering quality.

Execution Sample

SciPy

from sklearn.metrics import adjusted_rand_score
true = [0, 0, 1, 1, 2, 2]
pred = [0, 0, 2, 1, 2, 2]
score = adjusted_rand_score(true, pred)
print(score)

Calculate Adjusted Rand Index to compare true and predicted cluster labels.

Execution Table

Step	Action	Input	Intermediate Result	Output
1	Input true labels	[0, 0, 1, 1, 2, 2]	-	Stored true labels
2	Input predicted labels	[0, 0, 2, 1, 2, 2]	-	Stored predicted labels
3	Calculate contingency matrix	true & pred	[[2, 0, 0], [0, 1, 1], [0, 0, 2]]	Contingency matrix computed
4	Compute index components	contingency matrix	Sum combinations for pairs	Pairs counted
5	Calculate Adjusted Rand Index	index components	Adjusted for chance	Score = 0.5757575757575757
6	Print score	score	-	0.5757575757575757
7	End	-	-	Execution complete

💡 All steps completed, Adjusted Rand Index calculated and printed

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 5	Final
true	None	[0, 0, 1, 1, 2, 2]	[0, 0, 1, 1, 2, 2]	[0, 0, 1, 1, 2, 2]	[0, 0, 1, 1, 2, 2]	[0, 0, 1, 1, 2, 2]
pred	None	None	[0, 0, 2, 1, 2, 2]	[0, 0, 2, 1, 2, 2]	[0, 0, 2, 1, 2, 2]	[0, 0, 2, 1, 2, 2]
contingency_matrix	None	None	None	[[2, 0, 0], [0, 1, 1], [0, 0, 2]]	[[2, 0, 0], [0, 1, 1], [0, 0, 2]]	[[2, 0, 0], [0, 1, 1], [0, 0, 2]]
score	None	None	None	None	0.5757575757575757	0.5757575757575757

Key Moments - 3 Insights

Why is the Adjusted Rand Index not 1 even though some clusters match?

Because the metric adjusts for chance grouping, partial mismatches lower the score as shown in step 5 of the execution table.

What does the contingency matrix represent in clustering evaluation?

It counts how many points fall into each pair of true and predicted clusters, as shown in step 3 of the execution table.

Why do we need both true and predicted labels as inputs?

Because cluster evaluation metrics compare these two label sets to measure similarity, as shown in steps 1 and 2.

Visual Quiz - 3 Questions

Look at the execution table at step 5, what is the Adjusted Rand Index score calculated?

A 0.0

B 1.0

C 0.5757575757575757

D 0.85

💡 Hint

Refer to the 'Output' column in step 5 of the execution_table.

At which step is the contingency matrix computed?

A Step 3

B Step 2

C Step 4

D Step 5

💡 Hint

Check the 'Action' column in the execution_table for when the contingency matrix is created.

If the predicted labels were identical to true labels, what would the Adjusted Rand Index be?

A Negative

B Exactly 1

C Close to 0

D Undefined

💡 Hint

Adjusted Rand Index equals 1 when clustering matches perfectly, as explained in key_moments.

Concept Snapshot

Cluster evaluation metrics compare true and predicted cluster labels.
Common metrics: Adjusted Rand Index, Silhouette Score, Homogeneity.
Input: true labels and predicted labels.
Output: score indicating clustering quality.
Higher score usually means better clustering.
Use metrics to choose or tune clustering methods.

Full Transcript

Cluster evaluation metrics help us measure how well our clustering matches the true groups. We start with two lists: true labels and predicted cluster labels. We pick a metric like Adjusted Rand Index, which compares these labels and adjusts for chance. The code calculates a contingency matrix counting overlaps between true and predicted clusters. Then it computes the score, which ranges from -1 to 1, where 1 means perfect match. The example shows a score of about 0.58, meaning partial agreement. This process helps us judge clustering quality and improve models.

Practice

1. Which cluster evaluation metric is best used when you do NOT have true labels for your data?

easy

A. Adjusted Rand Index

B. Silhouette Score

C. Accuracy Score

D. Mean Squared Error

Solution

1. Step 1: Understand the role of true labels

   Adjusted Rand Index requires true labels to compare clusters, so it is not suitable without labels.

2. Step 2: Identify metrics for unknown labels

   Silhouette Score measures how well clusters are separated without needing true labels.

3. Final Answer:

   Silhouette Score -> Option B

4. Quick Check:

   Unknown labels = Silhouette Score [OK]

Hint: Use silhouette score when labels are unknown [OK]

Common Mistakes:

• Confusing Adjusted Rand Index as label-free
• Choosing accuracy score which needs labels
• Using mean squared error for clustering

2. Which of the following is the correct way to import the silhouette_score function for cluster evaluation?

easy

A. from scipy.cluster import silhouette_score

B. from scipy.spatial.distance import silhouette_score

C. from scipy.cluster.hierarchy import silhouette_score

D. from sklearn.metrics import silhouette_score

Solution

1. Step 1: Check common library modules

   Silhouette score is available in sklearn.metrics module, not in scipy.cluster or spatial.distance.

2. Step 2: Verify import syntax

   The correct import is from sklearn.metrics import silhouette_score.

3. Final Answer:

   from sklearn.metrics import silhouette_score -> Option D

4. Quick Check:

   Correct import = sklearn.metrics [OK]

Hint: Silhouette score is in sklearn.metrics module [OK]

Common Mistakes:

• Importing from scipy.cluster directly
• Using scipy.spatial.distance for silhouette_score
• Confusing hierarchy module with vq

3. What is the output of the following code snippet?

from scipy.cluster.vq import kmeans, vq, whiten
import numpy as np

data = np.array([[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]])
whitened = whiten(data)
centroids, _ = kmeans(whitened, 2)
cluster_labels, _ = vq(whitened, centroids)
from sklearn.metrics import silhouette_score
score = silhouette_score(whitened, cluster_labels)
print(round(score, 2))

medium

A. 0.75

B. 1.00

C. 0.35

D. 0.55

Solution

1. Step 1: Understand the code flow

   The code whitens data, runs kmeans for 2 clusters, assigns labels, then calculates silhouette score.

2. Step 2: Interpret silhouette score meaning

   Data clearly forms two groups; silhouette score is around 0.75 indicating good cluster separation.

3. Final Answer:

   0.75 -> Option A

4. Quick Check:

   Well-separated clusters ≈ 0.75 silhouette [OK]

Hint: Silhouette near 0.75 means good cluster separation [OK]

Common Mistakes:

• Expecting silhouette score of 1.0 always
• Confusing whitened data with original scale
• Misreading cluster labels

4. Identify the error in this code snippet for calculating Davies-Bouldin score:

from scipy.spatial.distance import davies_bouldin_score

labels = [0, 0, 1, 1]
data = [[1, 2], [1, 4], [10, 2], [10, 4]]
score = davies_bouldin_score(data, labels)
print(score)

medium

A. Data must be a numpy array, not list

B. Labels and data length mismatch

C. Importing davies_bouldin_score from wrong module

D. Davies-Bouldin score requires true labels

Solution

1. Step 1: Check import source

   Davies-Bouldin score is in sklearn.metrics, not scipy.spatial.distance.

2. Step 2: Validate data and labels

   Data and labels lengths match and data as list works with sklearn, so no error there.

3. Final Answer:

   Importing davies_bouldin_score from wrong module -> Option C

4. Quick Check:

   Correct import is sklearn.metrics [OK]

Hint: Davies-Bouldin score is in sklearn.metrics, not scipy [OK]

Common Mistakes:

• Importing from scipy.spatial.distance
• Assuming data must be numpy array
• Thinking Davies-Bouldin needs true labels

5. You have true labels and predicted cluster labels for a dataset. Which metric from scipy or sklearn should you use to evaluate clustering quality by comparing these labels?

hard

A. Adjusted Rand Index

B. Davies-Bouldin Score

C. Silhouette Score

D. Calinski-Harabasz Index

Solution

1. Step 1: Identify metrics needing true labels

   Adjusted Rand Index compares predicted clusters with true labels to measure similarity.

2. Step 2: Exclude label-free metrics

   Silhouette, Davies-Bouldin, and Calinski-Harabasz do not use true labels for evaluation.

3. Final Answer:

   Adjusted Rand Index -> Option A

4. Quick Check:

   True vs predicted labels = Adjusted Rand Index [OK]

Hint: Use Adjusted Rand Index to compare true and predicted labels [OK]

Common Mistakes:

• Using silhouette score with true labels
• Confusing Davies-Bouldin as label-based
• Choosing Calinski-Harabasz for label comparison