Pre-trained models (ResNet, VGG, EfficientNet) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Pre-trained models (ResNet, VGG, EfficientNet)

Which metric matters for Pre-trained Models and WHY

When using pre-trained models like ResNet, VGG, or EfficientNet, the key metrics to watch are accuracy and top-1/top-5 accuracy. These show how well the model predicts the correct class among its guesses. Since these models are often used for image classification, accuracy tells us how often the model gets the right answer.

Also, inference speed and model size matter because pre-trained models are used in real applications where speed and memory are important.

Confusion Matrix Example

      Actual \ Predicted | Cat | Dog | Bird
      -------------------|-----|-----|-----
      Cat                | 50  |  2  |  3
      Dog                |  4  | 45  |  1
      Bird               |  2  |  3  | 40
    

This matrix shows how many images of each actual class were predicted correctly or confused with others. For example, 50 cats were correctly predicted as cats (true positives for cat), while 2 cats were wrongly predicted as dogs (false negatives for cat).

Precision vs Recall Tradeoff

In image classification with pre-trained models, precision means how many predicted images of a class are actually correct. Recall means how many actual images of a class the model found.

For example, if you use a pre-trained model to detect rare animals, high recall is important so you don't miss any rare animals. But if you want to label images for a photo album, high precision is better to avoid wrong labels.

Good vs Bad Metric Values for Pre-trained Models

Good: Accuracy above 80% on a standard dataset like ImageNet means the model is performing well. Top-5 accuracy above 90% means the correct class is usually in the top guesses.

Bad: Accuracy below 50% or top-5 accuracy below 70% suggests the model is not learning well or the data is very different from the pre-trained data.

Common Pitfalls in Metrics for Pre-trained Models

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. For example, if 90% of images are cats, a model that always predicts cat gets 90% accuracy but is useless.
• Data leakage: Using test images in training inflates accuracy falsely.
• Overfitting: Very high training accuracy but low test accuracy means the model memorized training images but can't generalize.
• Ignoring inference speed: A model with great accuracy but very slow predictions may not be practical.

Self Check

Your pre-trained model has 98% accuracy but only 12% recall on a rare class like 'fraud' or 'disease'. Is it good for production?

Answer: No. The model misses most of the rare cases (low recall), which is dangerous. Even with high overall accuracy, it fails to detect important cases. You should improve recall for the rare class.

Key Result

Accuracy and recall are key metrics; high accuracy alone can be misleading if recall on important classes is low.

Practice

1. Which of the following is a key advantage of using pre-trained models like ResNet, VGG, or EfficientNet in computer vision tasks?

easy

A. They reduce the size of the input images automatically.

B. They save training time by using knowledge from large datasets.

C. They only work for text data, not images.

D. They always require training from scratch for every new task.

Solution

1. Step 1: Understand what pre-trained models do

   Pre-trained models are trained on large datasets and learn useful features that can be reused.

2. Step 2: Identify the benefit in context

   Using these models saves time because you don't need to train from scratch for every new task.

3. Final Answer:

   They save training time by using knowledge from large datasets. -> Option B

4. Quick Check:

   Pre-trained models save time = D [OK]

Hint: Pre-trained means already trained on big data [OK]

Common Mistakes:

• Thinking pre-trained models need full retraining
• Confusing image and text data applicability
• Assuming input size changes automatically

2. Which of the following is the correct way to load a pre-trained ResNet model in PyTorch?

easy

A. model = torch.load('resnet50')

B. model = torchvision.load_resnet50()

C. model = torchvision.models.ResNet50(weights='imagenet')

D. model = torchvision.models.resnet50(pretrained=True)

Solution

1. Step 1: Recall PyTorch syntax for loading pre-trained models

   In PyTorch, pre-trained models are loaded via torchvision.models with pretrained=True argument.

2. Step 2: Check each option

   model = torchvision.models.resnet50(pretrained=True) uses correct function and argument. Others are incorrect or invalid syntax.

3. Final Answer:

   model = torchvision.models.resnet50(pretrained=True) -> Option D

4. Quick Check:

   PyTorch pre-trained flag = pretrained=True [OK]

Hint: Use pretrained=True in torchvision.models [OK]

Common Mistakes:

• Using torch.load for model architecture
• Wrong function names like load_resnet50
• Incorrect argument names like weights='imagenet'

3. Consider this PyTorch code snippet using a pre-trained VGG16 model:

import torchvision.models as models
model = models.vgg16(pretrained=True)
print(type(model.features))

What will be the output type of model.features?

medium

A. <class 'torch.nn.Linear'>

B. <class 'torch.nn.ModuleList'>

C. <class 'torch.nn.Sequential'>

D. <class 'torch.nn.Conv2d'>

Solution

1. Step 1: Understand VGG16 model structure in PyTorch

   VGG16's feature extractor is implemented as a torch.nn.Sequential container of layers.

2. Step 2: Identify the type of model.features

   model.features groups convolutional layers in a Sequential module, so its type is torch.nn.Sequential.

3. Final Answer:

   <class 'torch.nn.Sequential'> -> Option C

4. Quick Check:

   VGG features = Sequential container [OK]

Hint: VGG features are in Sequential container [OK]

Common Mistakes:

• Confusing Sequential with ModuleList
• Thinking features is a single layer like Linear or Conv2d
• Not knowing PyTorch container types

4. You try to fine-tune a pre-trained EfficientNet model but get an error: AttributeError: module 'torchvision.models' has no attribute 'efficientnet'. What is the most likely cause?

medium

A. Your torchvision version is outdated and does not include EfficientNet.

B. You forgot to import torch.

C. EfficientNet models are not available in PyTorch.

D. You need to set pretrained=True to access EfficientNet.

Solution

1. Step 1: Understand the error message

   The error says torchvision.models has no attribute 'efficientnet', meaning the function is missing.

2. Step 2: Check common causes

   EfficientNet was added in newer torchvision versions. An outdated version lacks it.

3. Final Answer:

   Your torchvision version is outdated and does not include EfficientNet. -> Option A

4. Quick Check:

   Missing attribute = outdated torchvision [OK]

Hint: Check torchvision version for model availability [OK]

Common Mistakes:

• Assuming import torch fixes model availability
• Thinking EfficientNet is not in PyTorch at all
• Confusing pretrained flag with missing attribute

5. You want to build an image classifier for a small dataset with limited computing power. Which pre-trained model is the best choice to balance accuracy and efficiency?

hard

A. EfficientNet, because it scales well and is efficient for small data.

B. VGG16, because it is simple but very large and slow.

C. ResNet50, because it is very deep and accurate but heavy.

D. Train a new model from scratch for best results.

Solution

1. Step 1: Consider dataset size and computing power

   Small data and limited power require efficient models to avoid overfitting and long training.

2. Step 2: Compare model characteristics

   ResNet50 is accurate but heavy; VGG16 is large and slow; EfficientNet is designed for efficiency and good accuracy.

3. Step 3: Choose the best fit

   EfficientNet balances accuracy and efficiency, making it ideal for small datasets and limited resources.

4. Final Answer:

   EfficientNet, because it scales well and is efficient for small data. -> Option A

5. Quick Check:

   Efficiency + accuracy = EfficientNet [OK]

Hint: EfficientNet balances speed and accuracy well [OK]

Common Mistakes:

• Choosing heavy models for small data
• Ignoring efficiency for limited computing power
• Thinking training from scratch is always better