Replacing classifier head in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to replace the classifier head of a pretrained model with a new linear layer.

PyTorch

import torch.nn as nn
import torchvision
model = torchvision.models.resnet18(pretrained=True)
model.fc = nn.[1](512, 10)

Drag options to blanks, or click blank then click option'

AReLU

BConv2d

CBatchNorm2d

DLinear

Attempts:

3 left

💡 Hint

Common Mistakes

Using Conv2d instead of Linear for the classifier head.

Not matching the input feature size of the new layer.

The classifier head of ResNet18 is replaced by a linear layer with input features 512 and output features 10.

2fill in blank

medium

Complete the code to freeze all layers except the new classifier head.

PyTorch

for param in model.parameters():
    param.[1] = False
for param in model.fc.parameters():
    param.requires_grad = True

Drag options to blanks, or click blank then click option'

Arequires_grad

Bgrad

Crequires_grad_

Drequires_grad__

Attempts:

3 left

💡 Hint

Common Mistakes

Using incorrect attribute names like grad or requires_grad_.

Not freezing the pretrained layers before training.

Setting requires_grad to False freezes the parameters so they won't update during training.

3fill in blank

hard

Fix the error in the code to correctly replace the classifier head with a new linear layer.

PyTorch

import torchvision
import torch.nn as nn
model = torchvision.models.resnet50(pretrained=True)
model.fc = nn.Linear([1], 5)

Drag options to blanks, or click blank then click option'

A2048

B512

C1024

D256

Attempts:

3 left

💡 Hint

Common Mistakes

Using 512 which is the input size for ResNet18, not ResNet50.

Using arbitrary numbers without checking the model architecture.

ResNet50's classifier head input features size is 2048, so the new linear layer must match this.

4fill in blank

hard

Fill both blanks to create a new classifier head with dropout and linear layers.

PyTorch

model.fc = nn.Sequential(
    nn.Dropout(p=[1]),
    nn.Linear(512, [2])
)

Drag options to blanks, or click blank then click option'

A0.5

B10

C5

D0.3

Attempts:

3 left

💡 Hint

Common Mistakes

Using dropout probability greater than 1 or less than 0.

Mismatch between linear layer output size and number of classes.

Dropout with p=0.5 is common to reduce overfitting, and the linear layer outputs 10 classes.

5fill in blank

hard

Fill all three blanks to replace the classifier head and freeze pretrained layers except the new head.

PyTorch

import torchvision
import torch.nn as nn
model = torchvision.models.resnet34(pretrained=True)
for param in model.parameters():
    param.[1] = False
model.fc = nn.Linear([2], [3])
for param in model.fc.parameters():
    param.requires_grad = True

Drag options to blanks, or click blank then click option'

Arequires_grad

B512

C7

D1000

Attempts:

3 left

💡 Hint

Common Mistakes

Not freezing pretrained layers before training.

Using wrong input or output sizes for the linear layer.

Freezing pretrained layers requires setting requires_grad to False. ResNet34's fc input features are 512, and output is 7 classes.

Practice

1. What is the main reason to replace the classifier head in a pretrained PyTorch model?

easy

A. To adapt the model to a new task with different output classes

B. To speed up the training by removing layers

C. To reduce the model size by deleting layers

D. To change the input image size the model accepts

Solution

1. Step 1: Understand the classifier head role

   The classifier head is the last layer that decides the output classes based on learned features.

2. Step 2: Reason about adapting to new tasks

   Replacing the classifier head allows the model to output predictions for new classes different from the original training.

3. Final Answer:

   To adapt the model to a new task with different output classes -> Option A

4. Quick Check:

   Classifier head replacement = new task adaptation [OK]

Hint: Classifier head controls output classes, replace for new tasks [OK]

Common Mistakes:

• Thinking replacing head changes input size
• Assuming it reduces model size significantly
• Believing it speeds up training by removing layers

2. Which of the following is the correct way to replace the classifier head of a pretrained ResNet model in PyTorch for 10 output classes?

easy

A. model.fc = nn.Linear(2048, 10)

B. model.classifier = nn.Linear(2048, 10)

C. model.fc = nn.Linear(512, 10)

D. model.head = nn.Linear(512, 10)

Solution

1. Step 1: Identify ResNet classifier attribute

   ResNet models use model.fc as the classifier head.

2. Step 2: Check input feature size for ResNet

   ResNet50 and similar have 2048 features before the classifier, so input size is 2048.

3. Final Answer:

   model.fc = nn.Linear(2048, 10) -> Option A

4. Quick Check:

   ResNet classifier = model.fc with 2048 input features [OK]

Hint: ResNet classifier is model.fc with 2048 input features [OK]

Common Mistakes:

• Using wrong attribute like model.classifier or model.head
• Using wrong input size like 512 instead of 2048
• Confusing ResNet with other models like VGG

3. Given the code below, what will be the output shape of the model's final layer after replacement?

import torch
import torch.nn as nn
from torchvision import models

model = models.resnet18(pretrained=True)
model.fc = nn.Linear(512, 5)

input_tensor = torch.randn(1, 3, 224, 224)
output = model(input_tensor)
print(output.shape)

medium

A. torch.Size([1, 1000])

B. torch.Size([1, 512])

C. torch.Size([1, 5])

D. torch.Size([3, 224, 224])

Solution

1. Step 1: Understand the replaced classifier output size

   The new classifier layer outputs 5 values per input (5 classes).

2. Step 2: Check input batch size and output shape

   Input batch size is 1, so output shape is (1, 5).

3. Final Answer:

   torch.Size([1, 5]) -> Option C

4. Quick Check:

   Output shape = (batch_size, output_classes) = (1, 5) [OK]

Hint: Output shape matches batch size and new class count [OK]

Common Mistakes:

• Expecting original 1000 classes output
• Confusing feature size with output size
• Misreading input tensor shape as output

4. You tried replacing the classifier head of a pretrained model with model.fc = nn.Linear(1024, 10) but got a runtime error during training. What is the most likely cause?

medium

A. The model.fc attribute does not exist in pretrained models

B. The output size 10 is too large for the model

C. You forgot to call model.eval() before training

D. The input feature size 1024 does not match the model's actual output features

Solution

1. Step 1: Check input feature size for classifier

   The input size to the new Linear layer must match the output features of the previous layer.

2. Step 2: Identify mismatch causing runtime error

   If 1024 is incorrect, the model will raise size mismatch errors during forward pass.

3. Final Answer:

   The input feature size 1024 does not match the model's actual output features -> Option D

4. Quick Check:

   Input size mismatch causes runtime error [OK]

Hint: Match Linear input size to previous layer output features [OK]

Common Mistakes:

• Assuming output size causes error
• Confusing eval mode with training errors
• Thinking model.fc is missing in pretrained models

5. You want to fine-tune a pretrained ResNet50 on a dataset with 15 classes. Which code snippet correctly replaces the classifier head and freezes all layers except the new head?

hard

A. model = models.resnet50(pretrained=True)
model.fc = nn.Linear(2048, 15)
for param in model.parameters():
param.requires_grad = False

B. model = models.resnet50(pretrained=True)
for param in model.parameters():
param.requires_grad = False
model.fc = nn.Linear(2048, 15)

C. model = models.resnet50(pretrained=True)
for param in model.fc.parameters():
param.requires_grad = False
model.fc = nn.Linear(2048, 15)

D. model = models.resnet50(pretrained=True)
model.fc = nn.Linear(512, 15)
for param in model.parameters():
param.requires_grad = True

Solution

1. Step 1: Freeze all existing model parameters

   Set param.requires_grad = False for all parameters to prevent updates during training.

2. Step 2: Replace classifier head with correct input/output sizes

   ResNet50's classifier input size is 2048; output size is 15 for new classes.

3. Step 3: Ensure new head parameters are trainable

   By replacing model.fc after freezing, new layer parameters default to requires_grad=True.

4. Final Answer:

   Freeze all params, then replace head with nn.Linear(2048, 15) -> Option B

5. Quick Check:

   Freeze old layers, replace head with correct sizes [OK]

Hint: Freeze before replacing head to keep new layer trainable [OK]

Common Mistakes:

• Freezing after replacing head disables new layer training
• Using wrong input size 512 instead of 2048
• Not freezing any layers when fine-tuning