Cluster evaluation metrics in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Cluster evaluation metrics

Which metric matters for cluster evaluation and WHY

When we group data points into clusters, we want to know how good these groups are. Unlike labeled tasks, clusters have no fixed answers. So, we use special metrics to check if points in the same cluster are similar and points in different clusters are different.

Common metrics include:

• Silhouette Score: Measures how close each point is to its own cluster compared to other clusters. Higher means better grouping.
• Davies-Bouldin Index: Looks at cluster similarity and separation. Lower values mean better clusters.
• Calinski-Harabasz Index: Compares variance within clusters to variance between clusters. Higher is better.

These metrics help us decide if our clusters make sense without needing true labels.

Confusion matrix or equivalent visualization

For clustering, we don't have a confusion matrix like classification. Instead, we can visualize clusters with a dendrogram or scatter plot colored by cluster.

Cluster 1: ● ● ●
Cluster 2: ○ ○ ○
Cluster 3: ▲ ▲ ▲

Silhouette values per point:
Point 1: 0.7
Point 2: 0.6
Point 3: 0.8
...

Higher silhouette means better fit in cluster.
    

Precision vs Recall tradeoff (or equivalent) with concrete examples

Clustering doesn't have precision or recall directly. But we balance two ideas:

• Compactness: Points in the same cluster should be close (like friends sitting together).
• Separation: Different clusters should be far apart (like groups at different tables).

Improving compactness might reduce separation and vice versa. For example, if we make clusters very tight, some points might be left out or form many small clusters. If we make clusters very broad, different groups might mix.

What "good" vs "bad" metric values look like for clustering

• Silhouette Score: Good: close to 1 (clear clusters). Bad: near 0 or negative (overlapping clusters).
• Davies-Bouldin Index: Good: close to 0 (clusters well separated). Bad: large values (clusters overlap).
• Calinski-Harabasz Index: Good: high values (distinct clusters). Bad: low values (clusters not distinct).

Example: Silhouette = 0.75 means clusters are well formed. Silhouette = 0.1 means clusters are mixed.

Common pitfalls in cluster evaluation metrics

• Ignoring scale: Features with different scales can distort distances and metrics.
• Choosing wrong number of clusters: Metrics can mislead if cluster count is too high or low.
• Overfitting clusters: Too many clusters can give high scores but no real meaning.
• Data shape: Metrics assume spherical clusters; irregular shapes can confuse them.
• Comparing different algorithms: Metrics may favor some methods unfairly.

Self-check question

Your clustering model has a Silhouette Score of 0.2. Is this good? Why or why not?

Answer: A Silhouette Score of 0.2 is low, meaning clusters overlap a lot or are not well separated. This suggests the clustering is poor and may need adjustment like changing cluster count or features.

Key Result

Silhouette Score near 1 means clear clusters; near 0 or negative means poor clustering.