Freezing layers in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does it mean to 'freeze layers' in a neural network?

Freezing layers means stopping their weights from changing during training. This keeps the learned features fixed while training other parts of the model.

beginner

Why do we freeze layers when using a pre-trained model?

We freeze layers to keep the useful features learned from large data. This saves time and avoids losing good knowledge while training on new data.

beginner

How do you freeze layers in PyTorch?

Set the layer's parameters' requires_grad attribute to False. For example: <br>for param in model.layer.parameters():<br>   param.requires_grad = False

intermediate

What happens if you forget to freeze layers when fine-tuning?

The pre-trained weights may change too much, losing useful features. This can cause slower training or worse results.

intermediate

Can you unfreeze layers after freezing them? Why would you do this?

Yes, you can set requires_grad back to True. This is useful to fine-tune the whole model after training only some layers first.

What does setting param.requires_grad = False do in PyTorch?

A Prevents the parameter from updating during training

B Deletes the parameter from the model

C Initializes the parameter with zeros

D Makes the parameter trainable

Setting requires_grad to False tells PyTorch not to compute gradients for that parameter, so it won't update during training.

Why freeze layers in transfer learning?

A To remove layers from the model

B To keep learned features and reduce training time

C To make the model slower

D To increase the model size

Freezing layers keeps the useful features learned before and speeds up training on new tasks.

Which layers are usually frozen first in a convolutional neural network?

A All layers are frozen equally

B Last layers near the output

C Random layers

D Early layers near the input

Early layers learn basic features like edges and textures, so they are often frozen to keep these general features.

What is a common next step after freezing layers and training the new layers?

A Unfreeze some layers and fine-tune the whole model

B Delete the frozen layers

C Stop training immediately

D Freeze more layers

After training new layers, unfreezing some layers allows fine-tuning the whole model for better accuracy.

If you freeze all layers in a model, what will happen during training?

A The model will learn faster

B Only biases will update

C No weights will update, so the model won't learn

D The model will randomly change weights

Freezing all layers means no parameters update, so the model cannot learn from new data.

Explain in your own words what freezing layers means and why it is useful in transfer learning.

Think about how you keep some parts fixed while changing others.

Describe how you would freeze and then unfreeze layers in a PyTorch model during training.

Focus on the requires_grad attribute and training steps.