Feature extraction strategy in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of feature extraction with frozen layers

Consider a pretrained ResNet18 model in PyTorch. You freeze all layers except the last fully connected layer and pass a batch of images through it. What will be the shape of the output tensor?

PyTorch

import torch
import torchvision.models as models

model = models.resnet18(pretrained=True)
for param in model.parameters():
    param.requires_grad = False
model.fc = torch.nn.Linear(model.fc.in_features, 10)  # new output classes

inputs = torch.randn(8, 3, 224, 224)  # batch of 8 images
outputs = model(inputs)
print(outputs.shape)

A torch.Size([1, 10])

B torch.Size([8, 512])

C torch.Size([8, 1000])

D torch.Size([8, 10])

💡 Hint

Think about the batch size and the number of output classes after replacing the last layer.

The model outputs a tensor with shape (batch_size, number_of_classes). Since batch size is 8 and the last layer outputs 10 classes, the shape is (8, 10).

❓ Model Choice

intermediate

Best model choice for feature extraction on small dataset

You want to use feature extraction on a small image dataset with limited labels. Which pretrained model is generally best to start with for extracting features?

A A small pretrained MobileNetV2 model

B A pretrained VGG16 model

C A large pretrained ResNet50 model

D A pretrained Transformer-based model like ViT

💡 Hint

Consider model size and overfitting risk on small datasets.

MobileNetV2 is lightweight and less likely to overfit on small datasets, making it a good choice for feature extraction in such cases.

❓ Hyperparameter

advanced

Choosing learning rate for fine-tuning after feature extraction

After extracting features using a pretrained model and replacing the last layer, you want to fine-tune the model. Which learning rate setting is most appropriate to start with?

A A very high learning rate like 0.1

B A moderate learning rate like 0.01

C A very low learning rate like 1e-5

D No learning rate, freeze all layers

💡 Hint

Fine-tuning pretrained weights requires careful small updates.

A very low learning rate like 1e-5 helps fine-tune pretrained weights gently without destroying learned features.

❓ Metrics

advanced

Evaluating feature extraction effectiveness

You extracted features from a pretrained model and trained a classifier on top. Which metric best shows if feature extraction helped improve classification?

A Validation accuracy compared to training from scratch

B Training loss of the classifier

C Number of model parameters

D Time taken to train the classifier

💡 Hint

Think about comparing performance with and without feature extraction.

Validation accuracy compared to training from scratch shows if feature extraction improved generalization.

🔧 Debug

expert

Debugging feature extraction output mismatch

You extract features from a pretrained CNN by removing the last layer. But your extracted features have shape (8, 512, 1, 1) instead of expected (8, 512). What is the likely cause?

PyTorch

import torch
import torchvision.models as models

model = models.resnet18(pretrained=True)
# Attempt to remove last layer
model = torch.nn.Sequential(*list(model.children())[:-1])

inputs = torch.randn(8, 3, 224, 224)
features = model(inputs)
print(features.shape)

A The last layer was not removed properly; output is from the final FC layer

B The model output is flattened incorrectly, so shape is wrong

C Input batch size is wrong causing unexpected output shape

D The model expects different input image size

💡 Hint

Check if the output tensor is still 4D and needs flattening.

Removing the last layer returns a 4D tensor (batch, channels, 1, 1). Without flattening, shape appears as (8, 512, 1, 1). Proper flattening is needed.