Fine-tuning approach in Computer Vision - ML Experiment: Train & Evaluate

Experiment - Fine-tuning approach

Problem:We want to classify images of cats and dogs using a neural network. Currently, we use a pre-trained model but train all layers from scratch on a small dataset.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.85

Issue:The model overfits: training accuracy is very high but validation accuracy is low, showing poor generalization.

Your Task

Reduce overfitting by fine-tuning only the last layers of the pre-trained model and improve validation accuracy to above 85% while keeping training accuracy below 92%.

Use the same pre-trained model architecture (MobileNetV2).

Freeze the base layers and only train the top layers initially.

Use the same dataset and batch size.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Load pre-trained MobileNetV2 without top layers
base_model = MobileNetV2(input_shape=(160, 160, 3), include_top=False, weights='imagenet')
base_model.trainable = False  # Freeze base

# Add new classification head
model = models.Sequential([
    base_model,
    layers.GlobalAveragePooling2D(),
    layers.Dropout(0.3),
    layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Data augmentation
train_datagen = ImageDataGenerator(rescale=1./255, rotation_range=20, width_shift_range=0.2,
                                   height_shift_range=0.2, horizontal_flip=True, validation_split=0.2)

train_generator = train_datagen.flow_from_directory(
    'cats_and_dogs/train', target_size=(160, 160), batch_size=32, class_mode='binary', subset='training')

validation_generator = train_datagen.flow_from_directory(
    'cats_and_dogs/train', target_size=(160, 160), batch_size=32, class_mode='binary', subset='validation')

# Train only new layers
history = model.fit(train_generator, epochs=10, validation_data=validation_generator)

# Unfreeze some base layers for fine-tuning
base_model.trainable = True
for layer in base_model.layers[:-20]:
    layer.trainable = False

model.compile(optimizer=tf.keras.optimizers.Adam(1e-5), loss='binary_crossentropy', metrics=['accuracy'])

# Continue training
history_fine = model.fit(train_generator, epochs=10, validation_data=validation_generator)

Frozen the base MobileNetV2 layers to keep pre-trained features.

Added a new classification head with dropout to reduce overfitting.

Used data augmentation to increase data variety.

Trained only the new layers first, then unfroze last 20 base layers for fine-tuning with a low learning rate.

Results Interpretation

Before fine-tuning: Training accuracy was 98% but validation accuracy was only 70%, showing overfitting.

After fine-tuning: Training accuracy reduced to 90%, validation accuracy improved to 87%, and validation loss decreased significantly.

Freezing pre-trained layers and training only new layers helps prevent overfitting on small datasets. Fine-tuning some base layers with a low learning rate further improves validation performance by adapting features to the new task.

Bonus Experiment

Try using a different pre-trained model like EfficientNetB0 and compare validation accuracy and training time.

💡 Hint

EfficientNet models are efficient and may give better accuracy with fewer parameters. Use the same fine-tuning approach.

Practice

1. What is the main purpose of fine-tuning a pre-trained computer vision model?

easy

A. To adapt the model to a new task using less data and time

B. To train a model from scratch with a large dataset

C. To increase the size of the model for better accuracy

D. To remove layers from the model to make it smaller

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means starting from a model already trained on a related task.

2. Step 2: Identify the benefit

   This approach saves time and data by reusing learned features for a new task.

3. Final Answer:

   To adapt the model to a new task using less data and time -> Option A

4. Quick Check:

   Fine-tuning = adapt pre-trained model fast [OK]

Hint: Fine-tuning means reusing a model to learn new tasks faster [OK]

Common Mistakes:

• Thinking fine-tuning trains from scratch
• Assuming fine-tuning always increases model size
• Confusing fine-tuning with pruning layers

2. Which code snippet correctly freezes the layers of a PyTorch model before fine-tuning?

easy

A. for param in model.parameters(): param.requires_grad = False

B. model.freeze_layers()

C. model.trainable = False

D. for layer in model.layers: layer.trainable = True

Solution

1. Step 1: Recall PyTorch freezing syntax

   In PyTorch, freezing means setting requires_grad = False for parameters.

2. Step 2: Match code to syntax

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and disables gradient updates.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers = requires_grad False [OK]

Hint: Freeze layers by setting requires_grad = False in PyTorch [OK]

Common Mistakes:

• Using non-existent methods like freeze_layers()
• Setting model.trainable instead of parameters
• Confusing trainable True/False for freezing

3. Given this PyTorch code snippet for fine-tuning, what will be the output of print(sum(p.requires_grad for p in model.parameters())) after freezing layers?

medium

A. Raises an error

B. Number of all model parameters

C. 0

D. Number of unfrozen parameters

Solution

1. Step 1: Understand freezing effect on requires_grad

   Freezing sets requires_grad = False for all parameters.

2. Step 2: Calculate sum of requires_grad flags

   Since all are False, sum counts zero True values.

3. Final Answer:

   0 -> Option C

4. Quick Check:

   All frozen means requires_grad sum = 0 [OK]

Hint: Frozen layers have requires_grad = False, sum is zero [OK]

Common Mistakes:

• Assuming sum counts total parameters
• Thinking sum counts unfrozen parameters without freezing
• Expecting an error from requires_grad attribute

4. You tried fine-tuning but the model's accuracy did not improve. Which mistake could cause this?

medium

A. Using a pre-trained model instead of training from scratch

B. Freezing all layers and not unfreezing any

C. Adding more layers without training them

D. Using a very high learning rate during fine-tuning

Solution

1. Step 1: Identify learning rate impact

   A very high learning rate can cause unstable training and no improvement.

2. Step 2: Evaluate other options

   Freezing all layers prevents learning but usually keeps baseline accuracy; pre-trained models help; adding untrained layers alone doesn't prevent improvement if trained.

3. Final Answer:

   Using a very high learning rate during fine-tuning -> Option D

4. Quick Check:

   High learning rate = no improvement [OK]

Hint: Use smaller learning rates for fine-tuning to improve accuracy [OK]

Common Mistakes:

• Ignoring learning rate effects
• Assuming freezing all layers always improves
• Thinking training from scratch is better always

5. You want to fine-tune a pre-trained CNN for a new image classification task with 5 classes. Which sequence of steps is best practice?

hard

A. Train entire model from scratch with random weights for 5 classes

B. Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate

C. Replace final layer with 5 outputs and train all layers at once with high learning rate

D. Freeze final layer, train earlier layers only, then unfreeze final layer

Solution

1. Step 1: Replace final layer for new classes

   Adjust output layer to match 5 classes for the new task.

2. Step 2: Freeze old layers and train new layer first

   This preserves learned features and trains new output layer quickly.

3. Step 3: Unfreeze some layers and fine-tune with low learning rate

   This improves model performance by adapting features carefully without large updates.

4. Final Answer:

   Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate -> Option B

5. Quick Check:

   Stepwise fine-tuning with low LR = best practice [OK]

Hint: Freeze, replace output, train new layer, then unfreeze and fine-tune [OK]

Common Mistakes:

• Training all layers at once with high learning rate
• Training from scratch ignoring pre-trained weights
• Freezing final layer instead of earlier layers