Why CNNs detect spatial patterns in PyTorch - Why Metrics Matter

Metrics & Evaluation - Why CNNs detect spatial patterns

Which metric matters for this concept and WHY

For convolutional neural networks (CNNs) detecting spatial patterns, accuracy and loss during training are key metrics. Accuracy shows how well the model recognizes patterns in images or spatial data. Loss tells us how far off the model's predictions are from the true labels. Since CNNs focus on spatial features, metrics like precision and recall also matter when classes are imbalanced or some patterns are rare.

Confusion matrix or equivalent visualization (ASCII)

      Confusion Matrix Example:

          Predicted
          Cat   Dog
    True Cat  45    5
         Dog   3   47

    Here:
    - True Positives (TP) for Cat = 45
    - False Positives (FP) for Cat = 3
    - False Negatives (FN) for Cat = 5
    - True Negatives (TN) for Cat = 47

    This matrix helps us calculate precision and recall for each class.
    

Precision vs Recall tradeoff with concrete examples

Imagine a CNN detecting tumors in medical images (spatial patterns). Here, recall is very important because missing a tumor (false negative) is dangerous. We want the model to catch as many tumors as possible, even if it means some false alarms (lower precision).

On the other hand, if a CNN detects defects in manufactured parts, precision matters more. We want to avoid marking good parts as defective (false positives) to save costs.

Balancing precision and recall depends on the task and consequences of errors.

What "good" vs "bad" metric values look like for this use case

Good metrics:

• High accuracy (e.g., above 90%) on spatial pattern recognition tasks.
• Precision and recall both above 85%, showing balanced detection.
• Low loss values steadily decreasing during training.

Bad metrics:

• Accuracy near random guess (e.g., 50% for two classes).
• Very low recall (e.g., 20%) meaning many patterns missed.
• High loss or loss not improving, indicating poor learning.

Metrics pitfalls

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. For example, if 95% of images are background, a model always predicting background gets 95% accuracy but fails to detect patterns.
• Data leakage: If training and test data overlap, metrics look better but model won't generalize.
• Overfitting indicators: Training accuracy very high but test accuracy low means model memorizes training patterns but fails on new data.

Self-check question

Your CNN model for detecting spatial patterns has 98% accuracy but only 12% recall on the important class. Is it good for production? Why not?

Answer: No, it is not good. The low recall means the model misses most of the important patterns, even though overall accuracy is high. This can happen if the important class is rare and the model predicts mostly the other class. For production, recall must be higher to catch most patterns.

Key Result

For CNNs detecting spatial patterns, balanced precision and recall alongside accuracy best show model effectiveness.