Fine-tuning approach in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Fine-tuning Layers

In a fine-tuning approach for a convolutional neural network, which layers are typically retrained to adapt the model to a new task?

A No layers are retrained; the model is used as-is without any changes.

B Only the first convolutional layers are retrained, freezing the rest.

C Only the final classification layers are retrained, while earlier layers remain frozen.

D All layers are retrained from scratch with new random weights.

💡 Hint

Think about which parts of the model capture general features versus task-specific features.

Fine-tuning usually involves freezing early layers that capture general features and retraining the final layers to adapt to the new task.

❓ Predict Output

intermediate

Output Shape After Fine-tuning

Given a pretrained CNN model with an output layer of size 1000 classes, you replace the output layer with a new layer of size 10 classes for fine-tuning. What will be the output shape of the model for a batch of 32 images?

Computer Vision

import torch
import torch.nn as nn

class PretrainedModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.features = nn.Sequential(nn.Conv2d(3, 16, 3), nn.ReLU())
        self.classifier = nn.Linear(16 * 30 * 30, 1000)

    def forward(self, x):
        x = self.features(x)
        x = x.view(x.size(0), -1)
        x = self.classifier(x)
        return x

model = PretrainedModel()
model.classifier = nn.Linear(16 * 30 * 30, 10)  # Replace output layer
input_tensor = torch.randn(32, 3, 32, 32)
output = model(input_tensor)
output_shape = output.shape

A (1, 10)

B (32, 10)

C (32, 1000)

D (10, 32)

💡 Hint

Remember the batch size is the first dimension in PyTorch tensors.

The output shape is (batch_size, number_of_classes). After replacing the output layer to 10 classes, the output shape for 32 images is (32, 10).

❓ Hyperparameter

advanced

Choosing Learning Rate for Fine-tuning

When fine-tuning a pretrained model, which learning rate strategy is generally recommended?

A Use a much smaller learning rate than training from scratch to avoid destroying pretrained weights.

B Use a very large learning rate to quickly adapt the model to the new data.

C Use the same learning rate as training from scratch for consistency.

D Use zero learning rate to keep weights fixed.

💡 Hint

Think about how pretrained weights should be adjusted carefully.

A smaller learning rate helps preserve useful pretrained features while allowing gradual adaptation to new data.

❓ Metrics

advanced

Evaluating Fine-tuned Model Performance

After fine-tuning a model on a new dataset, which metric would best indicate if the model has successfully adapted without overfitting?

A Validation accuracy remains high and close to training accuracy.

B Training accuracy is very high but validation accuracy is very low.

C Training loss increases while validation loss decreases.

D Validation accuracy is zero.

💡 Hint

Consider what it means when training and validation metrics are similar.

High and similar training and validation accuracy indicate good generalization and successful fine-tuning.

🔧 Debug

expert

Identifying Fine-tuning Bug in Code

Consider this PyTorch code snippet for fine-tuning a pretrained model. What error will it raise?

Computer Vision

import torch
import torch.nn as nn
from torchvision import models

model = models.resnet18(pretrained=True)
for param in model.parameters():
    param.requires_grad = False

model.fc = nn.Linear(model.fc.in_features, 5)

optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# Training loop omitted

# Attempt to update only trainable parameters
optimizer.zero_grad()
loss = torch.tensor(1.0, requires_grad=True)
loss.backward()
optimizer.step()

A AttributeError because model.fc is not replaced correctly.

B RuntimeError because optimizer tries to update parameters with requires_grad=False.

C TypeError due to incorrect loss tensor creation.

D No error; optimizer updates only trainable parameters.

💡 Hint

Think about how PyTorch optimizers handle parameters with requires_grad=False.

PyTorch optimizers skip parameters where requires_grad=False, so no error occurs during optimizer.step().

Practice

1. What is the main purpose of fine-tuning a pre-trained computer vision model?

easy

A. To adapt the model to a new task using less data and time

B. To train a model from scratch with a large dataset

C. To increase the size of the model for better accuracy

D. To remove layers from the model to make it smaller

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means starting from a model already trained on a related task.

2. Step 2: Identify the benefit

   This approach saves time and data by reusing learned features for a new task.

3. Final Answer:

   To adapt the model to a new task using less data and time -> Option A

4. Quick Check:

   Fine-tuning = adapt pre-trained model fast [OK]

Hint: Fine-tuning means reusing a model to learn new tasks faster [OK]

Common Mistakes:

• Thinking fine-tuning trains from scratch
• Assuming fine-tuning always increases model size
• Confusing fine-tuning with pruning layers

2. Which code snippet correctly freezes the layers of a PyTorch model before fine-tuning?

easy

A. for param in model.parameters(): param.requires_grad = False

B. model.freeze_layers()

C. model.trainable = False

D. for layer in model.layers: layer.trainable = True

Solution

1. Step 1: Recall PyTorch freezing syntax

   In PyTorch, freezing means setting requires_grad = False for parameters.

2. Step 2: Match code to syntax

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and disables gradient updates.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers = requires_grad False [OK]

Hint: Freeze layers by setting requires_grad = False in PyTorch [OK]

Common Mistakes:

• Using non-existent methods like freeze_layers()
• Setting model.trainable instead of parameters
• Confusing trainable True/False for freezing

3. Given this PyTorch code snippet for fine-tuning, what will be the output of print(sum(p.requires_grad for p in model.parameters())) after freezing layers?

medium

A. Raises an error

B. Number of all model parameters

C. 0

D. Number of unfrozen parameters

Solution

1. Step 1: Understand freezing effect on requires_grad

   Freezing sets requires_grad = False for all parameters.

2. Step 2: Calculate sum of requires_grad flags

   Since all are False, sum counts zero True values.

3. Final Answer:

   0 -> Option C

4. Quick Check:

   All frozen means requires_grad sum = 0 [OK]

Hint: Frozen layers have requires_grad = False, sum is zero [OK]

Common Mistakes:

• Assuming sum counts total parameters
• Thinking sum counts unfrozen parameters without freezing
• Expecting an error from requires_grad attribute

4. You tried fine-tuning but the model's accuracy did not improve. Which mistake could cause this?

medium

A. Using a pre-trained model instead of training from scratch

B. Freezing all layers and not unfreezing any

C. Adding more layers without training them

D. Using a very high learning rate during fine-tuning

Solution

1. Step 1: Identify learning rate impact

   A very high learning rate can cause unstable training and no improvement.

2. Step 2: Evaluate other options

   Freezing all layers prevents learning but usually keeps baseline accuracy; pre-trained models help; adding untrained layers alone doesn't prevent improvement if trained.

3. Final Answer:

   Using a very high learning rate during fine-tuning -> Option D

4. Quick Check:

   High learning rate = no improvement [OK]

Hint: Use smaller learning rates for fine-tuning to improve accuracy [OK]

Common Mistakes:

• Ignoring learning rate effects
• Assuming freezing all layers always improves
• Thinking training from scratch is better always

5. You want to fine-tune a pre-trained CNN for a new image classification task with 5 classes. Which sequence of steps is best practice?

hard

A. Train entire model from scratch with random weights for 5 classes

B. Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate

C. Replace final layer with 5 outputs and train all layers at once with high learning rate

D. Freeze final layer, train earlier layers only, then unfreeze final layer

Solution

1. Step 1: Replace final layer for new classes

   Adjust output layer to match 5 classes for the new task.

2. Step 2: Freeze old layers and train new layer first

   This preserves learned features and trains new output layer quickly.

3. Step 3: Unfreeze some layers and fine-tune with low learning rate

   This improves model performance by adapting features carefully without large updates.

4. Final Answer:

   Freeze all layers, replace final layer with 5 outputs, train only final layer, then unfreeze some layers and fine-tune with low learning rate -> Option B

5. Quick Check:

   Stepwise fine-tuning with low LR = best practice [OK]

Hint: Freeze, replace output, train new layer, then unfreeze and fine-tune [OK]

Common Mistakes:

• Training all layers at once with high learning rate
• Training from scratch ignoring pre-trained weights
• Freezing final layer instead of earlier layers