Augmentation policy search (AutoAugment) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Augmentation policy search (AutoAugment)

Which metric matters for Augmentation policy search (AutoAugment) and WHY

AutoAugment helps improve image model training by finding the best ways to change images. The key metric to watch is validation accuracy. This shows if the model learns better with new image styles. Higher accuracy means the augmentation helps the model see more useful patterns and generalize well to new images.

Confusion matrix or equivalent visualization

Confusion Matrix Example (for image classification):

          Predicted
          Cat  Dog  Bird
Actual Cat  45   3    2
       Dog   4   40    6
       Bird  1    5   44

Total samples = 150

From this, accuracy = (45+40+44)/150 = 0.86 (86%)

AutoAugment aims to improve this accuracy by better training data.
    

Precision vs Recall tradeoff with concrete examples

In image classification, precision and recall matter per class. For example, if the model detects cats:

• Precision means: Of all images predicted as cats, how many really are cats?
• Recall means: Of all actual cat images, how many did the model find?

AutoAugment helps balance these by making the model see varied images. For example, if recall is low, the model misses many cats. Augmentation can add more cat-like images to fix this. If precision is low, the model confuses cats with dogs. Augmentation can help by showing clearer differences.

What "good" vs "bad" metric values look like for this use case

Good: Validation accuracy above 85% with balanced precision and recall near 0.8 or higher per class. This means the model correctly identifies most images and makes few mistakes.

Bad: Validation accuracy below 70%, or precision/recall below 0.5 for important classes. This means the model struggles to learn useful features, possibly due to poor augmentation or overfitting.

Metrics pitfalls

• Accuracy paradox: High accuracy can hide poor class performance if classes are imbalanced.
• Data leakage: Augmentation must be applied only on training data, not validation, or metrics will be misleading.
• Overfitting indicators: Training accuracy much higher than validation accuracy means augmentation may not be effective or model is too complex.

Self-check question

Your image classifier trained with AutoAugment has 98% training accuracy but only 60% validation accuracy. Is this good? Why or why not?

Answer: This is not good. The large gap shows overfitting. The model memorizes training images but fails to generalize. AutoAugment should help reduce this gap by making training images more varied. You may need to adjust augmentation policies or model complexity.

Key Result

Validation accuracy is key to measure if AutoAugment improves model generalization by providing better training images.