Albumentations integration in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Albumentations integration

Which metric matters for Albumentations integration and WHY

Albumentations is a tool to change images before training. It helps the model see many versions of the same picture. This makes the model better at understanding new pictures. The key metric to watch is validation accuracy or validation loss. This shows if the model is learning well on new, unseen images. If accuracy goes up or loss goes down on validation data, Albumentations is helping.

Confusion matrix example

Imagine a model classifying cats and dogs. After training with Albumentations, the confusion matrix might look like this:

          Predicted Cat   Predicted Dog
Actual Cat       45             5
Actual Dog        7            43

Total samples = 100

Here, true positives (TP) for cats = 45, false positives (FP) for cats = 7, false negatives (FN) for cats = 5, true negatives (TN) for cats = 43.

Precision vs Recall tradeoff with Albumentations

Albumentations helps the model see more varied images, which can improve both precision and recall.

• Precision means when the model says "this is a cat," how often it is right.
• Recall means how many actual cats the model finds.

For example, if Albumentations creates many good variations, the model learns better features and can find more cats (higher recall) without making many mistakes (high precision).

Good vs Bad metric values for Albumentations use

Good: Validation accuracy improves by 3-5% compared to no augmentation. Precision and recall both above 0.8. Loss decreases steadily.

Bad: Validation accuracy stays the same or drops. Precision or recall below 0.5. Loss fluctuates or increases, showing model confusion.

Common pitfalls with Albumentations metrics

• Overfitting: Training accuracy very high but validation accuracy low means augmentation may be insufficient or incorrect.
• Data leakage: Augmenting validation data by mistake inflates validation accuracy falsely.
• Wrong augmentations: Using augmentations that distort images too much can confuse the model, hurting metrics.
• Ignoring metric trends: Only looking at training loss without validation metrics can hide problems.

Self-check question

Your model trained with Albumentations has 98% training accuracy but only 12% recall on the cat class in validation. Is this good?

Answer: No. The model is not finding most cats in new images (low recall). This means it misses many cats, so Albumentations or training needs adjustment.

Key Result

Validation accuracy and recall are key to check if Albumentations helps the model learn better on new images.