Fine-tuning approach in Computer Vision

Introduction

Fine-tuning helps a model learn new tasks faster by starting from a model that already knows something similar.

You want to teach a model to recognize new types of images but have only a small dataset.

You want to improve an existing model's accuracy on a specific task.

You want to save time and computing power by not training a model from scratch.

You want to adapt a general model to a special use case, like medical images.

You want to use a pre-trained model as a starting point for your own project.

Syntax

Computer Vision

1. Load a pre-trained model.
2. Freeze some layers to keep old knowledge.
3. Replace or add new layers for your task.
4. Train the new layers on your data.
5. Optionally unfreeze some layers and train more.

Freezing layers means their weights do not change during training.

Replacing the last layer is common to match the number of classes in your task.

Examples

This example loads MobileNetV2 pre-trained on ImageNet, freezes all layers, and adds a new output layer for 5 classes.

Computer Vision

from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model

base_model = MobileNetV2(weights='imagenet', include_top=False, pooling='avg')
for layer in base_model.layers:
    layer.trainable = False

output = Dense(5, activation='softmax')(base_model.output)
model = Model(inputs=base_model.input, outputs=output)

This example unfreezes the last 10 layers to fine-tune them on new data.

Computer Vision

for layer in model.layers[-10:]:
    layer.trainable = True

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, epochs=5)

Sample Model

This program shows how to fine-tune a pre-trained MobileNetV2 model on a small dummy dataset with 5 classes. It first trains only the new output layer, then unfreezes some layers to improve learning.

Computer Vision

import tensorflow as tf
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical
import numpy as np

# Create dummy data: 100 images 96x96x3, 5 classes
x_train = np.random.rand(100, 96, 96, 3).astype('float32')
y_train = to_categorical(np.random.randint(5, size=100), num_classes=5)

# Load pre-trained MobileNetV2 without top layers
base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(96,96,3), pooling='avg')

# Freeze base model layers
for layer in base_model.layers:
    layer.trainable = False

# Add new output layer for 5 classes
output = Dense(5, activation='softmax')(base_model.output)
model = Model(inputs=base_model.input, outputs=output)

# Compile model
model.compile(optimizer=Adam(), loss='categorical_crossentropy', metrics=['accuracy'])

# Train only new layers
history = model.fit(x_train, y_train, epochs=3, batch_size=10, verbose=2)

# Unfreeze last 20 layers for fine-tuning
for layer in base_model.layers[-20:]:
    layer.trainable = True

# Recompile with lower learning rate
model.compile(optimizer=Adam(1e-5), loss='categorical_crossentropy', metrics=['accuracy'])

# Continue training
history_fine = model.fit(x_train, y_train, epochs=2, batch_size=10, verbose=2)

Important Notes

Fine-tuning works best when your new task is similar to the original task the model was trained on.

Start by training only new layers, then gradually unfreeze more layers to avoid losing old knowledge.

Use a smaller learning rate when fine-tuning to make small adjustments.

Summary

Fine-tuning reuses a pre-trained model to learn new tasks faster.

Freeze old layers first, then train new layers, and finally unfreeze some layers to improve.

Use smaller learning rates during fine-tuning for better results.

Practice

1. What is the main purpose of fine-tuning a pre-trained computer vision model?

easy

A. To adapt the model to a new task using less data and time

B. To train a model from scratch with a large dataset

C. To increase the size of the model for better accuracy

D. To remove layers from the model to make it smaller

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means starting from a model already trained on a related task.

2. Step 2: Identify the benefit

   This approach saves time and data by reusing learned features for a new task.

3. Final Answer:

   To adapt the model to a new task using less data and time -> Option A

4. Quick Check:

   Fine-tuning = adapt pre-trained model fast [OK]

Hint: Fine-tuning means reusing a model to learn new tasks faster [OK]

Common Mistakes:

• Thinking fine-tuning trains from scratch
• Assuming fine-tuning always increases model size
• Confusing fine-tuning with pruning layers

2. Which code snippet correctly freezes the layers of a PyTorch model before fine-tuning?

easy

A. for param in model.parameters(): param.requires_grad = False

B. model.freeze_layers()

C. model.trainable = False

D. for layer in model.layers: layer.trainable = True

Solution

1. Step 1: Recall PyTorch freezing syntax

   In PyTorch, freezing means setting requires_grad = False for parameters.

2. Step 2: Match code to syntax

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and disables gradient updates.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers = requires_grad False [OK]

Hint: Freeze layers by setting requires_grad = False in PyTorch [OK]

Common Mistakes:

• Using non-existent methods like freeze_layers()
• Setting model.trainable instead of parameters
• Confusing trainable True/False for freezing

3. Given this PyTorch code snippet for fine-tuning, what will be the output of print(sum(p.requires_grad for p in model.parameters())) after freezing layers?

medium

A. Raises an error

B. Number of all model parameters

C. 0

D. Number of unfrozen parameters

Solution

1. Step 1: Understand freezing effect on requires_grad

   Freezing sets requires_grad = False for all parameters.

2. Step 2: Calculate sum of requires_grad flags

   Since all are False, sum counts zero True values.

3. Final Answer:

   0 -> Option C

4. Quick Check:

   All frozen means requires_grad sum = 0 [OK]

Hint: Frozen layers have requires_grad = False, sum is zero [OK]

Common Mistakes:

• Assuming sum counts total parameters
• Thinking sum counts unfrozen parameters without freezing
• Expecting an error from requires_grad attribute

4. You tried fine-tuning but the model's accuracy did not improve. Which mistake could cause this?

medium

A. Using a pre-trained model instead of training from scratch

B. Freezing all layers and not unfreezing any

C. Adding more layers without training them

D. Using a very high learning rate during fine-tuning

Solution

1. Step 1: Identify learning rate impact

   A very high learning rate can cause unstable training and no improvement.

2. Step 2: Evaluate other options

   Freezing all layers prevents learning but usually keeps baseline accuracy; pre-trained models help; adding untrained layers alone doesn't prevent improvement if trained.

3. Final Answer:

   Using a very high learning rate during fine-tuning -> Option D

4. Quick Check:

   High learning rate = no improvement [OK]

Hint: Use smaller learning rates for fine-tuning to improve accuracy [OK]

Common Mistakes:

• Ignoring learning rate effects
• Assuming freezing all layers always improves
• Thinking training from scratch is better always

5. You want to fine-tune a pre-trained CNN for a new image classification task with 5 classes. Which sequence of steps is best practice?

hard

A. Train entire model from scratch with random weights for 5 classes

B. Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate

C. Replace final layer with 5 outputs and train all layers at once with high learning rate

D. Freeze final layer, train earlier layers only, then unfreeze final layer

Solution

1. Step 1: Replace final layer for new classes

   Adjust output layer to match 5 classes for the new task.

2. Step 2: Freeze old layers and train new layer first

   This preserves learned features and trains new output layer quickly.

3. Step 3: Unfreeze some layers and fine-tune with low learning rate

   This improves model performance by adapting features carefully without large updates.

4. Final Answer:

   Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate -> Option B

5. Quick Check:

   Stepwise fine-tuning with low LR = best practice [OK]

Hint: Freeze, replace output, train new layer, then unfreeze and fine-tune [OK]

Common Mistakes:

• Training all layers at once with high learning rate
• Training from scratch ignoring pre-trained weights
• Freezing final layer instead of earlier layers