Freezing and unfreezing layers in TensorFlow - Model Pipeline Trace

Model Pipeline - Freezing and unfreezing layers

This pipeline shows how a model uses frozen layers to keep learned features fixed, then unfreezes layers to fine-tune and improve accuracy.

Data Flow - 6 Stages

1Data input

1000 rows x 32 x 32 x 3→Load images with labels→1000 rows x 32 x 32 x 3

Image of a cat with label 'cat'

↓

2Preprocessing

1000 rows x 32 x 32 x 3→Normalize pixel values to 0-1 range→1000 rows x 32 x 32 x 3

Pixel value 120 becomes 0.47

↓

3Feature extraction (frozen layers)

1000 rows x 32 x 32 x 3→Pass through frozen convolutional layers→1000 rows x 8 x 8 x 64

Feature map highlighting edges

↓

4Train classifier layers

1000 rows x 8 x 8 x 64→Train dense layers on extracted features→1000 rows x 10 (class probabilities)

[0.1, 0.7, 0.05, ..., 0.02]

↓

5Unfreeze layers

1000 rows x 8 x 8 x 64→Unfreeze some convolutional layers for fine-tuning→1000 rows x 8 x 8 x 64

Layers now trainable to adjust features

↓

6Fine-tune entire model

1000 rows x 8 x 8 x 64→Train all layers with lower learning rate→1000 rows x 10 (class probabilities)

[0.05, 0.8, 0.03, ..., 0.01]

Training Trace - Epoch by Epoch

Epoch 1: ************ (loss=1.2)
Epoch 2: ******** (loss=0.9)
Epoch 3: ******* (loss=0.7)
Epoch 4: ****** (loss=0.65)
Epoch 5: **** (loss=0.5)
Epoch 6: *** (loss=0.45)

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.55	Training classifier layers with frozen base
2	0.9	0.68	Accuracy improves as classifier learns
3	0.7	0.75	Stable improvement with frozen base
4	0.65	0.78	Unfreeze some layers for fine-tuning
5	0.5	0.85	Fine-tuning improves feature extraction
6	0.45	0.88	Model converges with unfreezing

Prediction Trace - 5 Layers

Layer 1: Input layer

Layer 2: Frozen convolutional layers

Layer 3: Trainable dense layers

Layer 4: Unfreeze and fine-tune convolutional layers

Layer 5: Final dense layers

Model Quiz - 3 Questions

Why do we freeze layers before training the classifier layers?

A To keep learned features fixed and speed up training

B To prevent the model from learning anything new

C To make the model ignore the input data

D To randomly change weights in the frozen layers

Key Insight

Freezing layers helps keep useful features stable while training new parts. Unfreezing allows the model to improve by adjusting earlier layers carefully, leading to better accuracy.