Albumentations library in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Albumentations library

Which metric matters for Albumentations library and WHY

Albumentations helps improve model performance by changing images in smart ways before training. The key metrics to watch are validation accuracy and validation loss. These show if the model learns better with augmented images. Also, robustness metrics like accuracy on new or noisy images matter because augmentation aims to make models strong against changes.

Confusion matrix example

      Actual \ Predicted | Cat | Dog | Rabbit
      -------------------------------------
      Cat               | 45  |  3  |   2
      Dog               |  4  | 40  |   6
      Rabbit            |  1  |  5  |  44
    

This confusion matrix shows how well the model classifies images after using Albumentations. More correct predictions (diagonal) mean better augmentation helped the model learn.

Precision vs Recall tradeoff with Albumentations

When using Albumentations, you want to keep a balance between precision and recall. For example, if you augment images too much, the model might get confused and lower precision (more false alarms). If you augment too little, recall might drop (missing real cases). Good augmentation finds a sweet spot to keep both high.

Example: In a dog detector, high precision means fewer wrong dog labels, high recall means catching most dogs. Albumentations helps by showing varied dog images during training.

What good vs bad metric values look like with Albumentations

• Good: Validation accuracy above 85%, balanced precision and recall above 80%, and stable loss decrease during training.
• Bad: Validation accuracy below 70%, big gap between precision and recall (like 90% vs 50%), or validation loss increasing (overfitting or bad augmentation).

Good metrics mean Albumentations helped the model learn useful features. Bad metrics suggest augmentation was too weak, too strong, or not suitable.

Common pitfalls when evaluating Albumentations impact

• Accuracy paradox: High accuracy but poor recall on rare classes means augmentation missed important cases.
• Data leakage: Using augmented versions of test images in training inflates metrics falsely.
• Overfitting: If validation loss rises while training loss falls, augmentation might be too weak or inconsistent.
• Wrong augmentation: Using unrealistic transformations can confuse the model and hurt metrics.

Self-check question

Your model trained with Albumentations has 98% accuracy but only 12% recall on the rare class. Is it good?

Answer: No. The low recall means the model misses most rare cases, which is bad especially if those cases matter. High accuracy can be misleading if the rare class is small. You should adjust augmentation or model to improve recall.

Key Result

Albumentations improves model robustness by enhancing validation accuracy and balancing precision-recall, but watch for overfitting and data leakage.

Practice

1. What is the main purpose of the Albumentations library in computer vision?

easy

A. To apply image transformations that help AI models learn better

B. To train deep learning models directly

C. To label images for datasets

D. To evaluate model accuracy

Solution

1. Step 1: Understand Albumentations role

   Albumentations is designed to change images by applying transformations like flips, crops, or color changes.

2. Step 2: Connect transformations to model training

   These changes create many versions of images, helping AI models learn better by seeing varied data.

3. Final Answer:

   To apply image transformations that help AI models learn better -> Option A

4. Quick Check:

   Albumentations = Image transformations for training [OK]

Hint: Albumentations = image changes to improve training [OK]

Common Mistakes:

• Thinking Albumentations trains models directly
• Confusing it with labeling tools
• Assuming it measures accuracy

2. Which of the following is the correct way to import the Compose function from Albumentations?

easy

A. from albumentations import Compose

B. import albumentations.Compose

C. from albumentations.Compose import *

D. import Compose from albumentations

Solution

1. Step 1: Recall Python import syntax

   To import a function from a module, use 'from module import function'.

2. Step 2: Match with Albumentations usage

   Albumentations provides Compose as a function inside the albumentations module, so 'from albumentations import Compose' is correct.

3. Final Answer:

   from albumentations import Compose -> Option A

4. Quick Check:

   Correct import syntax = from albumentations import Compose [OK]

Hint: Use 'from module import function' for imports [OK]

Common Mistakes:

• Using dot notation in import incorrectly
• Trying to import with 'import Compose from ...'
• Using wildcard import from Compose

3. What will be the output shape of the image after applying this Albumentations transform?

import albumentations as A
transform = A.Compose([A.Resize(128, 128)])
augmented = transform(image=image)
result = augmented['image'].shape

medium

A. (256, 256, 3)

B. (128, 128)

C. (3, 128, 128)

D. (128, 128, 3)

Solution

1. Step 1: Understand Resize transform

   The Resize transform changes the image size to the given height and width, here 128x128 pixels.

2. Step 2: Check image shape format

   Albumentations uses height, width, and channels format (H, W, C). Assuming input image has 3 color channels, output shape is (128, 128, 3).

3. Final Answer:

   (128, 128, 3) -> Option D

4. Quick Check:

   Resize to 128x128 with 3 channels = (128,128,3) [OK]

Hint: Resize sets height and width; channels stay same [OK]

Common Mistakes:

• Confusing channel order with PyTorch (C,H,W)
• Ignoring color channels in output shape
• Assuming Resize doubles size

4. Identify the error in this Albumentations code snippet:

import albumentations as A
transform = A.Compose([
    A.HorizontalFlip(p=0.5),
    A.RandomBrightnessContrast(p=1.2)
])
augmented = transform(image=image)

medium

A. Compose cannot take a list of transforms

B. HorizontalFlip does not accept probability parameter

C. Probability p=1.2 is invalid; must be between 0 and 1

D. Missing import for RandomBrightnessContrast

Solution

1. Step 1: Check probability parameters

   Albumentations expects probability values (p) between 0 and 1 for transforms.

2. Step 2: Identify invalid p value

   RandomBrightnessContrast has p=1.2, which is invalid and will cause an error.

3. Final Answer:

   Probability p=1.2 is invalid; must be between 0 and 1 -> Option C

4. Quick Check:

   Probability p must be ≤ 1 [OK]

Hint: Probabilities must be between 0 and 1 [OK]

Common Mistakes:

• Setting p > 1 by mistake
• Thinking HorizontalFlip lacks p parameter
• Assuming Compose needs special import

5. You want to create a training pipeline that randomly flips images horizontally 50% of the time and then applies either a random brightness or contrast change (but not both) with equal chance. Which Albumentations Compose setup achieves this?

hard

A. A.Compose([A.OneOf([A.HorizontalFlip(p=0.5), A.RandomBrightness(p=1), A.RandomContrast(p=1)], p=1)])

B. A.Compose([A.HorizontalFlip(p=0.5), A.OneOf([A.RandomBrightness(p=1), A.RandomContrast(p=1)], p=1)])

C. A.Compose([A.HorizontalFlip(p=0.5), A.RandomBrightness(p=0.5), A.RandomContrast(p=0.5)])

D. A.Compose([A.HorizontalFlip(p=1), A.RandomBrightness(p=0.5), A.RandomContrast(p=0.5)])

Solution

1. Step 1: Understand HorizontalFlip probability

   We want horizontal flip with 50% chance, so use A.HorizontalFlip(p=0.5).

2. Step 2: Apply either brightness or contrast randomly

   Use A.OneOf with two transforms inside, each with p=1, and OneOf's p=1 to ensure exactly one is applied.

3. Step 3: Check options

   A.Compose([A.HorizontalFlip(p=0.5), A.OneOf([A.RandomBrightness(p=1), A.RandomContrast(p=1)], p=1)]) matches this logic exactly. Others either apply both brightness and contrast or misuse probabilities.

4. Final Answer:

   A.Compose([A.HorizontalFlip(p=0.5), A.OneOf([A.RandomBrightness(p=1), A.RandomContrast(p=1)], p=1)]) -> Option B

5. Quick Check:

   Use OneOf for exclusive choice [OK]

Hint: Use OneOf to pick one transform randomly [OK]

Common Mistakes:

• Applying brightness and contrast together
• Setting wrong probabilities in OneOf
• Using OneOf to include HorizontalFlip