Data augmentation importance in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Data augmentation importance

Which metric matters for Data Augmentation and WHY

Data augmentation helps models see more varied examples by changing images slightly. This usually improves accuracy and generalization. We focus on validation accuracy and validation loss to check if the model learns well on new, unseen images. Higher accuracy and lower loss on validation data mean augmentation is helping the model avoid overfitting and perform better in real life.

Confusion Matrix Example

Imagine a model classifying images into cats and dogs. After training with augmentation, the confusion matrix might look like this:

      | Predicted Cat | Predicted Dog |
      |--------------|---------------|
      | True Cat: 45 | False Dog: 5  |
      | False Cat: 3 | True Dog: 47  |
    

Total samples = 45 + 5 + 3 + 47 = 100

From this, we calculate:

• Precision (Cat) = 45 / (45 + 3) = 0.94
• Recall (Cat) = 45 / (45 + 5) = 0.90
• Accuracy = (45 + 47) / 100 = 0.92

This shows the model is good at recognizing cats and dogs after augmentation.

Precision vs Recall Tradeoff with Data Augmentation

Data augmentation can help balance precision and recall by making the model robust to variations.

• High Precision, Low Recall: Model is very sure when it predicts a class but misses many true cases. For example, it only labels very clear cat images as cats, missing some cats that look different.
• High Recall, Low Precision: Model finds most cats but sometimes mistakes dogs for cats.

Augmentation helps increase both by showing the model many versions of cats and dogs, so it learns to recognize them better in different conditions.

Good vs Bad Metric Values for Data Augmentation

Good:

• Validation accuracy improves or stays stable compared to no augmentation.
• Validation loss decreases, showing better learning on new data.
• Balanced precision and recall above 85% for key classes.

Bad:

• Validation accuracy drops significantly, meaning augmentation is hurting learning.
• Validation loss increases or fluctuates wildly.
• Precision or recall very low, showing model confusion.

Common Pitfalls in Metrics with Data Augmentation

• Overfitting despite augmentation: Augmentation is not a fix-all; if the model is too complex, it can still memorize training data.
• Data leakage: Augmented images too similar to validation images can give false high accuracy.
• Ignoring class imbalance: Augmentation might increase some classes more than others, skewing metrics.
• Accuracy paradox: High accuracy can hide poor performance on rare classes; always check precision and recall.

Self Check

Your model trained with data augmentation shows 98% accuracy but only 12% recall on a rare class like fraud detection. Is it good?

Answer: No, it is not good. The low recall means the model misses most fraud cases, which is critical. High accuracy is misleading because most data is non-fraud. You need to improve recall, possibly by better augmentation or other techniques.

Key Result

Data augmentation improves validation accuracy and recall by exposing the model to varied data, helping it generalize better.