Freezing layers in PyTorch - Model Metrics & Evaluation

Metrics & Evaluation - Freezing layers

Which metric matters for Freezing layers and WHY

When freezing layers in a model, the main goal is to keep learned features fixed while training new parts. The key metrics to watch are validation loss and validation accuracy. These show if the model is learning new tasks well without forgetting old knowledge. If validation loss decreases and accuracy improves, freezing is helping. If not, the frozen layers might block learning.

Confusion matrix example

    Confusion Matrix (after freezing some layers):

          Predicted
          Pos   Neg
    Actual
    Pos   85    15
    Neg   10    90

    TP = 85, FP = 10, TN = 90, FN = 15

    Precision = 85 / (85 + 10) = 0.895
    Recall = 85 / (85 + 15) = 0.85
    F1 Score = 2 * (0.895 * 0.85) / (0.895 + 0.85) ≈ 0.872
    

This shows the model still predicts well after freezing layers, balancing precision and recall.

Precision vs Recall tradeoff with Freezing layers

Freezing layers can limit how much the model adapts. This might keep precision high by avoiding false alarms but reduce recall if the model misses new patterns.

For example, in a spam filter, freezing early layers keeps known spam features fixed, so precision stays high (few good emails marked spam). But recall might drop if new spam types appear and the model can't learn them well.

Choosing which layers to freeze balances this tradeoff: freeze too many and recall drops, freeze too few and training is slower or unstable.

What "good" vs "bad" metric values look like for Freezing layers

Good: Validation accuracy improves or stays stable, validation loss decreases, and precision and recall remain balanced after freezing layers.

Bad: Validation accuracy drops, loss stays high or increases, or recall drops sharply indicating the model can't learn new patterns due to frozen layers.

Common pitfalls when evaluating Freezing layers

• Overfitting: If only final layers train, they might overfit small new data while frozen layers don't adapt.
• Data leakage: Using test data during tuning can falsely show good metrics.
• Ignoring validation loss: Accuracy alone can hide if model is not improving properly.
• Freezing too many layers: Can block learning new features, hurting recall and overall performance.

Self-check question

Your model has 98% accuracy but only 12% recall on fraud detection after freezing layers. Is it good for production? Why or why not?

Answer: No, it is not good. High accuracy can be misleading if fraud cases are rare. Low recall means the model misses most frauds, which is dangerous. The frozen layers might prevent learning new fraud patterns, so you should adjust which layers to freeze or retrain more.

Key Result

Validation accuracy and recall are key to check if freezing layers helps learning without losing important new patterns.