Data transforms in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Data transforms

Which metric matters for Data transforms and WHY

Data transforms prepare raw data for the model. The key metric to check is model accuracy or loss after applying transforms. Good transforms help the model learn better by making data consistent and easier to understand.

For example, normalizing images to a common scale helps the model focus on patterns, not brightness differences. So, the metric to watch is how well the model performs after transforms, usually accuracy or loss.

Confusion matrix or equivalent visualization

Confusion Matrix Example (after applying data transforms):

          Predicted
          0    1
Actual 0 50   10
       1  5   35

- True Positives (TP) = 35
- True Negatives (TN) = 50
- False Positives (FP) = 10
- False Negatives (FN) = 5

Total samples = 50 + 10 + 5 + 35 = 100

This matrix shows how well the model predicts after data transforms.
    

Precision vs Recall tradeoff with concrete examples

Data transforms can affect precision and recall by changing data quality.

Precision means how many predicted positives are actually correct.

Recall means how many actual positives the model finds.

Example: If transforms remove noise well, precision improves because fewer wrong positives happen.

If transforms lose some important data, recall drops because the model misses real positives.

Good transforms balance precision and recall by cleaning data without losing important info.

What "good" vs "bad" metric values look like for Data transforms

Good: After transforms, model accuracy is high (e.g., >85%), loss is low, and confusion matrix shows few errors.

Bad: Accuracy drops or loss increases after transforms, or confusion matrix shows many false positives or false negatives.

This means transforms may have distorted data or removed useful information.

Metrics pitfalls

• Accuracy paradox: High accuracy can hide poor performance if data is imbalanced.
• Data leakage: Applying transforms using test data info can inflate metrics falsely.
• Overfitting indicators: If metrics improve on training but worsen on validation, transforms might cause overfitting.
• Inconsistent transforms: Applying different transforms to train and test data causes bad metrics.

Self-check question

Your model has 98% accuracy but only 12% recall on fraud cases after applying data transforms. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most fraud cases, which is dangerous. Even with high accuracy, the model fails to find fraud. The transforms might have removed important fraud signals or caused imbalance. You need to improve recall for this use case.

Key Result

Data transforms impact model accuracy and loss; good transforms improve these metrics by making data consistent and meaningful.

Practice

1. What is the main purpose of using data transforms in PyTorch?

easy

A. To prepare and modify raw data before feeding it to a model

B. To create new neural network layers

C. To evaluate model accuracy

D. To save the trained model to disk

Solution

1. Step 1: Understand the role of data transforms

   Data transforms modify raw data to make it suitable for model input, such as resizing or normalizing images.

2. Step 2: Differentiate from other tasks

   Creating layers, evaluating accuracy, or saving models are separate tasks unrelated to data transforms.

3. Final Answer:

   To prepare and modify raw data before feeding it to a model -> Option A

4. Quick Check:

   Data transforms = prepare data [OK]

Hint: Transforms prepare data, not model or evaluation tasks [OK]

Common Mistakes:

• Confusing transforms with model building
• Thinking transforms evaluate model performance
• Mixing transforms with saving models

2. Which of the following is the correct way to chain multiple transforms using PyTorch's torchvision?

easy

A. transforms.Sequence(transforms.Resize(256), transforms.ToTensor())

B. transforms.Chain([transforms.Resize(256), transforms.ToTensor()])

C. transforms.Pipeline(transforms.Resize(256), transforms.ToTensor())

D. transforms.Compose([transforms.Resize(256), transforms.ToTensor()])

Solution

1. Step 1: Recall the correct class for chaining transforms

   PyTorch uses transforms.Compose to chain multiple transforms in a list.

2. Step 2: Check syntax correctness

   Only transforms.Compose accepts a list of transforms; others like Sequence, Pipeline, Chain do not exist in torchvision.

3. Final Answer:

   transforms.Compose([transforms.Resize(256), transforms.ToTensor()]) -> Option D

4. Quick Check:

   Use Compose for chaining transforms [OK]

Hint: Remember Compose takes a list of transforms [OK]

Common Mistakes:

• Using non-existent classes like Sequence or Pipeline
• Passing transforms as separate arguments instead of a list
• Confusing Compose with other transform functions

3. What will be the output shape of the tensor after applying the following transform to a 3-channel image of size 100x100?
transform = transforms.Compose([transforms.Resize(50), transforms.ToTensor()])

medium

A. [1, 50, 50]

B. [50, 50, 3]

C. [3, 50, 50]

D. [50, 3, 50]

Solution

1. Step 1: Understand Resize effect

   transforms.Resize(50) resizes the image to 50x50 pixels, keeping the number of channels unchanged (3 channels).

2. Step 2: Understand ToTensor output shape

   transforms.ToTensor() converts the image to a tensor with shape [channels, height, width], so shape is [3, 50, 50].

3. Final Answer:

   [3, 50, 50] -> Option C

4. Quick Check:

   Tensor shape = [channels, height, width] [OK]

Hint: Tensor shape is channels first after ToTensor [OK]

Common Mistakes:

• Assuming channels last format
• Confusing Resize parameter as channels
• Mixing height and width order

4. Identify the error in this transform code snippet:

transform = transforms.Compose([
    transforms.Resize(128),
    transforms.Normalize(mean=[0.5], std=[0.5]),
    transforms.ToTensor()
])

medium

A. ToTensor should be first

B. Normalize should come after ToTensor

C. Resize must be after Normalize

D. No error, code is correct

Solution

1. Step 1: Check order of transforms

   transforms.Normalize expects a tensor input, but here it is before ToTensor, which converts image to tensor.

2. Step 2: Correct order

   ToTensor must come before Normalize to convert image to tensor first, then normalize it.

3. Final Answer:

   Normalize should come after ToTensor -> Option B

4. Quick Check:

   Normalize after ToTensor [OK]

Hint: Normalize needs tensor input, so place after ToTensor [OK]

Common Mistakes:

• Placing Normalize before ToTensor
• Assuming Resize order matters with Normalize
• Thinking ToTensor can be anywhere

5. You want to apply random horizontal flip and then normalize an image tensor. Which of the following transform sequences is correct to ensure proper data augmentation and normalization?

hard

A. transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor(), transforms.Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5])])

B. transforms.Compose([transforms.ToTensor(), transforms.RandomHorizontalFlip(), transforms.Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5])])

C. transforms.Compose([transforms.Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5]), transforms.RandomHorizontalFlip(), transforms.ToTensor()])

D. transforms.Compose([transforms.RandomHorizontalFlip(), transforms.Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5]), transforms.ToTensor()])

Solution

1. Step 1: Understand transform order for augmentation and normalization

   RandomHorizontalFlip works on PIL images, so it must come before ToTensor.

2. Step 2: Normalize after converting to tensor

   Normalize requires tensor input, so it must come after ToTensor.

3. Final Answer:

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor(), transforms.Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5])]) -> Option A

4. Quick Check:

   Flip before ToTensor, Normalize after ToTensor [OK]

Hint: Augment on images, normalize on tensors [OK]

Common Mistakes:

• Applying Normalize before ToTensor
• Applying RandomHorizontalFlip after ToTensor
• Mixing order of augmentation and normalization