Pre-trained models (ResNet, VGG, EfficientNet) in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Pre-trained models (ResNet, VGG, EfficientNet)

Which metric matters for Pre-trained Models and WHY

When using pre-trained models like ResNet, VGG, or EfficientNet, the key metrics to watch are accuracy and top-1/top-5 accuracy. These show how well the model predicts the correct class among its guesses. Since these models are often used for image classification, accuracy tells us how often the model gets the right answer.

Also, inference speed and model size matter because pre-trained models are used in real applications where speed and memory are important.

Confusion Matrix Example

      Actual \ Predicted | Cat | Dog | Bird
      -------------------|-----|-----|-----
      Cat                | 50  |  2  |  3
      Dog                |  4  | 45  |  1
      Bird               |  2  |  3  | 40
    

This matrix shows how many images of each actual class were predicted correctly or confused with others. For example, 50 cats were correctly predicted as cats (true positives for cat), while 2 cats were wrongly predicted as dogs (false negatives for cat).

Precision vs Recall Tradeoff

In image classification with pre-trained models, precision means how many predicted images of a class are actually correct. Recall means how many actual images of a class the model found.

For example, if you use a pre-trained model to detect rare animals, high recall is important so you don't miss any rare animals. But if you want to label images for a photo album, high precision is better to avoid wrong labels.

Good vs Bad Metric Values for Pre-trained Models

Good: Accuracy above 80% on a standard dataset like ImageNet means the model is performing well. Top-5 accuracy above 90% means the correct class is usually in the top guesses.

Bad: Accuracy below 50% or top-5 accuracy below 70% suggests the model is not learning well or the data is very different from the pre-trained data.

Common Pitfalls in Metrics for Pre-trained Models

• Accuracy paradox: High accuracy can be misleading if classes are imbalanced. For example, if 90% of images are cats, a model that always predicts cat gets 90% accuracy but is useless.
• Data leakage: Using test images in training inflates accuracy falsely.
• Overfitting: Very high training accuracy but low test accuracy means the model memorized training images but can't generalize.
• Ignoring inference speed: A model with great accuracy but very slow predictions may not be practical.

Self Check

Your pre-trained model has 98% accuracy but only 12% recall on a rare class like 'fraud' or 'disease'. Is it good for production?

Answer: No. The model misses most of the rare cases (low recall), which is dangerous. Even with high overall accuracy, it fails to detect important cases. You should improve recall for the rare class.

Key Result

Accuracy and recall are key metrics; high accuracy alone can be misleading if recall on important classes is low.