Freezing layers in PyTorch - Model Pipeline Trace

Model Pipeline - Freezing layers

This pipeline shows how freezing layers in a neural network helps keep some parts fixed while training others. It speeds up training and preserves learned features.

Data Flow - 4 Stages

1Load dataset

1000 rows x 3 channels x 32 height x 32 width→Load images and labels→1000 samples x 3 x 32 x 32

Image tensor with pixel values and label 'cat'

↓

2Preprocessing

1000 samples x 3 x 32 x 32→Normalize pixel values to 0-1 range→1000 samples x 3 x 32 x 32

Pixel value 120 scaled to 0.47

↓

3Feature extraction (frozen layers)

1000 samples x 3 x 32 x 32→Pass through frozen convolutional layers→1000 samples x 16 x 28 x 28

Feature map highlighting edges

↓

4Trainable layers

1000 samples x 16 x 28 x 28→Pass through trainable fully connected layers→1000 samples x 10 classes

Output logits for 10 classes

Training Trace - Epoch by Epoch

Loss
1.2 |****
0.9 |***
0.7 |**
0.6 |*
0.55|*
    +---------
    Epochs 1-5

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Loss starts high, accuracy low as model begins learning
2	0.9	0.60	Loss decreases, accuracy improves as trainable layers adjust
3	0.7	0.72	Continued improvement, frozen layers keep features stable
4	0.6	0.78	Model converging, trainable layers fine-tuned
5	0.55	0.82	Training stabilizes with good accuracy

Prediction Trace - 5 Layers

Layer 1: Input image

Layer 2: Frozen convolutional layers

Layer 3: Trainable fully connected layers

Layer 4: Softmax activation

Layer 5: Prediction

Model Quiz - 3 Questions

Why do we freeze some layers during training?

A To keep learned features unchanged and speed up training

B To make the model train slower

C To increase the number of trainable parameters

D To randomly change weights

Key Insight

Freezing layers lets the model keep useful features fixed while training other parts. This helps faster learning and avoids losing previously learned knowledge.