Data augmentation with transforms in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Data augmentation with transforms

Which metric matters for Data Augmentation and WHY

Data augmentation helps models learn better by showing more varied examples. The key metric to watch is validation accuracy or validation loss. This shows if the model is truly learning to generalize to new data, not just memorizing training data.

We focus on validation metrics because augmentation aims to reduce overfitting. If validation accuracy improves or validation loss decreases after augmentation, it means the transforms helped the model learn better.

Confusion Matrix Example

For classification tasks, a confusion matrix shows how well the model predicts each class after augmentation.

      Actual \ Predicted | Cat | Dog | Rabbit
      -------------------|-----|-----|-------
      Cat                | 45  | 3   | 2     
      Dog                | 4   | 43  | 3     
      Rabbit             | 1   | 2   | 47    
    

This matrix sums to 150 samples. From it, we calculate precision, recall, and F1 for each class to see if augmentation improved class-wise predictions.

Precision vs Recall Tradeoff with Augmentation

Augmentation can affect precision and recall differently. For example, if augmentation adds noise, the model might become more cautious, increasing precision but lowering recall.

Imagine a dog detector: high recall means catching most dogs, but low precision means some cats are wrongly labeled as dogs. Augmentation should balance this by improving recall without hurting precision too much.

Good vs Bad Metric Values for Augmentation

Good: Validation accuracy increases by a few points, validation loss decreases, and class-wise precision and recall improve or stay stable.

Bad: Validation accuracy stays the same or drops, validation loss increases, or precision/recall drop significantly, indicating augmentation harmed learning.

Common Pitfalls in Metrics with Augmentation

• Accuracy paradox: Accuracy may look good if classes are imbalanced, hiding poor performance on rare classes.
• Data leakage: Augmentation applied before splitting data can leak test info into training, inflating metrics.
• Overfitting indicators: If training accuracy is very high but validation accuracy is low, augmentation might be insufficient or wrongly applied.

Self Check

Your model has 98% accuracy but only 12% recall on fraud cases after augmentation. Is it good?

Answer: No. Despite high accuracy, the model misses most fraud cases (low recall). For fraud detection, recall is critical to catch fraud. Augmentation or model needs improvement to raise recall.

Key Result

Validation accuracy and recall are key to evaluate if data augmentation helps the model generalize better.

Practice

1. What is the main purpose of using transforms.Compose in PyTorch data augmentation?

easy

A. To combine multiple image transformations into one pipeline

B. To train the model faster by skipping data loading

C. To convert images into numpy arrays

D. To save the augmented images to disk automatically

Solution

1. Step 1: Understand the role of transforms.Compose

   transforms.Compose is used to chain several image transformations so they apply sequentially to the input image.

2. Step 2: Identify the correct purpose

   It does not speed up training directly, convert images to numpy, or save images. Its main job is combining transformations.

3. Final Answer:

   To combine multiple image transformations into one pipeline -> Option A

4. Quick Check:

   transforms.Compose = combine transforms [OK]

Hint: Remember Compose chains transforms in order [OK]

Common Mistakes:

• Thinking Compose speeds up training
• Confusing Compose with image saving
• Assuming Compose converts image formats

2. Which of the following is the correct way to apply a horizontal flip and convert an image to a tensor using PyTorch transforms?

easy

A. transforms.Compose(transforms.RandomHorizontalFlip(), transforms.ToTensor())

B. transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()])

C. transforms.ToTensor(transforms.RandomHorizontalFlip())

D. transforms.RandomHorizontalFlip(transforms.ToTensor())

Solution

1. Step 1: Check the syntax for combining transforms

   PyTorch requires transforms to be passed as a list inside transforms.Compose([]).

2. Step 2: Validate each option

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()]) correctly uses a list inside Compose. The other options misuse function calls or pass arguments incorrectly.

3. Final Answer:

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()]) -> Option B

4. Quick Check:

   Compose needs list of transforms [OK]

Hint: Use Compose with a list of transforms inside brackets [OK]

Common Mistakes:

• Passing transforms as separate arguments instead of a list
• Calling transforms inside each other incorrectly
• Forgetting to convert images to tensor

3. Given the following code, what will be the shape of the output tensor after applying the transforms to a 3-channel 64x64 image?

transform = transforms.Compose([
    transforms.RandomRotation(90),
    transforms.ToTensor()
])
output = transform(image)

medium

A. [1, 64, 64]

B. [64, 64, 3]

C. [3, 64, 64]

D. [64, 3, 64]

Solution

1. Step 1: Understand the transform effects

   RandomRotation rotates the image but does not change its size or channels. ToTensor converts the image to a tensor with shape [channels, height, width].

2. Step 2: Determine output shape

   Input image is 3 channels, 64x64 pixels. After ToTensor, shape is [3, 64, 64]. Rotation keeps size same.

3. Final Answer:

   [3, 64, 64] -> Option C

4. Quick Check:

   ToTensor output shape = [channels, height, width] [OK]

Hint: ToTensor outputs [channels, height, width] shape [OK]

Common Mistakes:

• Confusing channel position in tensor shape
• Assuming rotation changes image size
• Thinking output is a numpy array shape

4. Identify the error in this PyTorch transform pipeline:

transform = transforms.Compose([
    transforms.RandomCrop(32),
    transforms.ToTensor,
    transforms.Normalize((0.5,), (0.5,))
])

medium

A. transforms.ToTensor is missing parentheses to call it

B. Normalize mean and std should be lists, not tuples

C. RandomCrop size should be a tuple, not an integer

D. Compose should not be used with Normalize

Solution

1. Step 1: Check each transform usage

   RandomCrop accepts an integer for size, so that is correct. Normalize accepts tuples for mean and std, so that is correct.

2. Step 2: Identify the missing parentheses

   transforms.ToTensor is a class, but it must be called as transforms.ToTensor() to create the transform instance.

3. Final Answer:

   transforms.ToTensor is missing parentheses to call it -> Option A

4. Quick Check:

   Call ToTensor() with parentheses [OK]

Hint: Always call transforms with parentheses [OK]

Common Mistakes:

• Forgetting parentheses on transform classes
• Thinking Normalize needs lists instead of tuples
• Misunderstanding RandomCrop size argument

5. You want to augment your training images by randomly flipping horizontally, rotating by up to 30 degrees, and normalizing with mean=0.5 and std=0.5 for each channel. Which transform pipeline correctly applies these steps in PyTorch?

hard

A. transforms.Compose([transforms.ToTensor(), transforms.RandomHorizontalFlip(), transforms.RandomRotation(30), transforms.Normalize((0.5,), (0.5,))])

B. transforms.Compose([transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), transforms.RandomHorizontalFlip(), transforms.RandomRotation(30), transforms.ToTensor()])

C. transforms.Compose([transforms.RandomRotation(30), transforms.RandomHorizontalFlip(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)), transforms.ToTensor()])

D. transforms.Compose([transforms.RandomHorizontalFlip(), transforms.RandomRotation(30), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

Solution

1. Step 1: Order of transforms matters

   Data augmentation like flipping and rotation must happen before converting to tensor. Normalization happens after ToTensor.

2. Step 2: Check each option's order and parameters

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.RandomRotation(30), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]) applies flip and rotation first, then ToTensor, then Normalize with correct mean/std for 3 channels. Others have wrong order or missing steps.

3. Final Answer:

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.RandomRotation(30), transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]) -> Option D

4. Quick Check:

   Augment before ToTensor, normalize after [OK]

Hint: Augment first, then ToTensor, then Normalize [OK]

Common Mistakes:

• Normalizing before ToTensor
• Applying augmentations after ToTensor
• Using wrong mean/std shapes for Normalize