Fine-tuning approach in Computer Vision - Model Pipeline Trace

Model Pipeline - Fine-tuning approach

This pipeline shows how a pre-trained computer vision model is adapted to a new task by fine-tuning. It starts with input images, processes them through a pre-trained model, then retrains some layers on new data to improve accuracy for the new task.

Data Flow - 5 Stages

1Input images

1000 images x 224 x 224 x 3→Raw images loaded and resized to 224x224 pixels with 3 color channels (RGB)→1000 images x 224 x 224 x 3

Image of a cat resized to 224x224 pixels

↓

2Preprocessing

1000 images x 224 x 224 x 3→Normalize pixel values to range 0-1→1000 images x 224 x 224 x 3

Pixel values scaled from 0-255 to 0-1

↓

3Feature extraction with pre-trained model

1000 images x 224 x 224 x 3→Pass images through pre-trained convolutional layers (frozen weights)→1000 images x 7 x 7 x 512

Feature map representing edges and textures

↓

4Fine-tuning layers

1000 images x 7 x 7 x 512→Unfreeze last convolutional block and retrain on new data→1000 images x 7 x 7 x 512 (updated weights)

Model adjusts filters to better detect new task features

↓

5Classification head

1000 images x 7 x 7 x 512→Flatten features and pass through dense layers to output class probabilities→1000 images x 10 classes

Output probabilities for 10 object categories

Training Trace - Epoch by Epoch

Epoch 1: ************ (loss=1.2)
Epoch 2: *********    (loss=0.9)
Epoch 3: *******      (loss=0.7)
Epoch 4: ******       (loss=0.6)
Epoch 5: *****        (loss=0.55)

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.55	Initial fine-tuning starts with moderate accuracy and high loss
2	0.9	0.68	Loss decreases and accuracy improves as model adapts
3	0.7	0.75	Continued improvement shows effective fine-tuning
4	0.6	0.80	Model learns task-specific features better
5	0.55	0.83	Training converges with good accuracy and low loss

Prediction Trace - 5 Layers

Layer 1: Input image

Layer 2: Pre-trained convolutional layers

Layer 3: Fine-tuned convolutional block

Layer 4: Flatten and dense layers

Layer 5: Final prediction

Model Quiz - 3 Questions

Why do we freeze most layers of the pre-trained model during fine-tuning?

A To speed up training by skipping all layers

B To keep learned general features and only adapt specific layers

C Because frozen layers improve accuracy automatically

D To prevent the model from making any changes

Key Insight

Fine-tuning leverages existing knowledge from a pre-trained model and adapts it to a new task by retraining only some layers. This approach saves time and data while improving accuracy for the new problem.

Practice

1. What is the main purpose of fine-tuning a pre-trained computer vision model?

easy

A. To adapt the model to a new task using less data and time

B. To train a model from scratch with a large dataset

C. To increase the size of the model for better accuracy

D. To remove layers from the model to make it smaller

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means starting from a model already trained on a related task.

2. Step 2: Identify the benefit

   This approach saves time and data by reusing learned features for a new task.

3. Final Answer:

   To adapt the model to a new task using less data and time -> Option A

4. Quick Check:

   Fine-tuning = adapt pre-trained model fast [OK]

Hint: Fine-tuning means reusing a model to learn new tasks faster [OK]

Common Mistakes:

• Thinking fine-tuning trains from scratch
• Assuming fine-tuning always increases model size
• Confusing fine-tuning with pruning layers

2. Which code snippet correctly freezes the layers of a PyTorch model before fine-tuning?

easy

A. for param in model.parameters(): param.requires_grad = False

B. model.freeze_layers()

C. model.trainable = False

D. for layer in model.layers: layer.trainable = True

Solution

1. Step 1: Recall PyTorch freezing syntax

   In PyTorch, freezing means setting requires_grad = False for parameters.

2. Step 2: Match code to syntax

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and disables gradient updates.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers = requires_grad False [OK]

Hint: Freeze layers by setting requires_grad = False in PyTorch [OK]

Common Mistakes:

• Using non-existent methods like freeze_layers()
• Setting model.trainable instead of parameters
• Confusing trainable True/False for freezing

3. Given this PyTorch code snippet for fine-tuning, what will be the output of print(sum(p.requires_grad for p in model.parameters())) after freezing layers?

medium

A. Raises an error

B. Number of all model parameters

C. 0

D. Number of unfrozen parameters

Solution

1. Step 1: Understand freezing effect on requires_grad

   Freezing sets requires_grad = False for all parameters.

2. Step 2: Calculate sum of requires_grad flags

   Since all are False, sum counts zero True values.

3. Final Answer:

   0 -> Option C

4. Quick Check:

   All frozen means requires_grad sum = 0 [OK]

Hint: Frozen layers have requires_grad = False, sum is zero [OK]

Common Mistakes:

• Assuming sum counts total parameters
• Thinking sum counts unfrozen parameters without freezing
• Expecting an error from requires_grad attribute

4. You tried fine-tuning but the model's accuracy did not improve. Which mistake could cause this?

medium

A. Using a pre-trained model instead of training from scratch

B. Freezing all layers and not unfreezing any

C. Adding more layers without training them

D. Using a very high learning rate during fine-tuning

Solution

1. Step 1: Identify learning rate impact

   A very high learning rate can cause unstable training and no improvement.

2. Step 2: Evaluate other options

   Freezing all layers prevents learning but usually keeps baseline accuracy; pre-trained models help; adding untrained layers alone doesn't prevent improvement if trained.

3. Final Answer:

   Using a very high learning rate during fine-tuning -> Option D

4. Quick Check:

   High learning rate = no improvement [OK]

Hint: Use smaller learning rates for fine-tuning to improve accuracy [OK]

Common Mistakes:

• Ignoring learning rate effects
• Assuming freezing all layers always improves
• Thinking training from scratch is better always

5. You want to fine-tune a pre-trained CNN for a new image classification task with 5 classes. Which sequence of steps is best practice?

hard

A. Train entire model from scratch with random weights for 5 classes

B. Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate

C. Replace final layer with 5 outputs and train all layers at once with high learning rate

D. Freeze final layer, train earlier layers only, then unfreeze final layer

Solution

1. Step 1: Replace final layer for new classes

   Adjust output layer to match 5 classes for the new task.

2. Step 2: Freeze old layers and train new layer first

   This preserves learned features and trains new output layer quickly.

3. Step 3: Unfreeze some layers and fine-tune with low learning rate

   This improves model performance by adapting features carefully without large updates.

4. Final Answer:

   Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate -> Option B

5. Quick Check:

   Stepwise fine-tuning with low LR = best practice [OK]

Hint: Freeze, replace output, train new layer, then unfreeze and fine-tune [OK]

Common Mistakes:

• Training all layers at once with high learning rate
• Training from scratch ignoring pre-trained weights
• Freezing final layer instead of earlier layers