Freezing layers in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to freeze all parameters in the model.

PyTorch

for param in model.parameters():
    param.[1] = False

Drag options to blanks, or click blank then click option'

Arequires_grad

Bgrad

Cfreeze

Ddetach

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'grad' instead of 'requires_grad'.

Trying to call a method like 'freeze()' which doesn't exist.

Setting requires_grad = False tells PyTorch not to update these parameters during training, effectively freezing them.

2fill in blank

medium

Complete the code to freeze only the first layer of the model.

PyTorch

for param in model.[1].parameters():
    param.requires_grad = False

Drag options to blanks, or click blank then click option'

Alayers

Bmodules

Cchildren

Dlayer1

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'children' or 'modules' which return iterables, not a single layer.

Using 'layers' which is not a standard attribute.

Accessing model.layer1 targets the first layer to freeze its parameters.

3fill in blank

hard

Fix the error in the code to freeze the model's convolutional layers only.

PyTorch

for name, param in model.named_parameters():
    if 'conv' in name:
        param.[1] = False

Drag options to blanks, or click blank then click option'

Agrad

Brequires_grad

Cdetach

Dfreeze

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'detach' which returns a new tensor but does not freeze parameters.

Trying to set 'grad' or 'freeze' which are invalid.

To freeze parameters, set requires_grad = False. Other options are incorrect or invalid.

4fill in blank

hard

Fill both blanks to freeze all layers except the last fully connected layer.

PyTorch

for name, param in model.named_parameters():
    if not [1] in name:
        param.[2] = False

Drag options to blanks, or click blank then click option'

A'fc'

Brequires_grad

C'conv'

Dgrad

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'conv' instead of 'fc' to exclude the last layer.

Trying to set 'grad' instead of 'requires_grad'.

We skip freezing parameters whose name contains 'fc' (fully connected layer). For others, set requires_grad = False to freeze.

5fill in blank

hard

Fill all three blanks to freeze all layers except those containing 'bn' or 'fc' in their names.

PyTorch

for name, param in model.named_parameters():
    if not ([1] in name or [2] in name):
        param.[3] = False

Drag options to blanks, or click blank then click option'

A'bn'

B'fc'

Crequires_grad

D'conv'

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'conv' instead of 'bn' or 'fc' to exclude layers.

Trying to set 'grad' or other invalid attributes.

We exclude layers with 'bn' (batch norm) and 'fc' (fully connected) from freezing. For others, set requires_grad = False.

Practice

1. What does freezing layers in a PyTorch model do during training?

easy

A. Removes the layers from the model

B. Increases the learning rate for those layers

C. Stops the layers' weights from updating

D. Duplicates the layers for faster training

Solution

1. Step 1: Understand freezing layers meaning

   Freezing layers means preventing their weights from changing during training.

2. Step 2: Effect on training

   When frozen, layers do not update weights, so they keep learned features intact.

3. Final Answer:

   Stops the layers' weights from updating -> Option C

4. Quick Check:

   Freezing = no weight updates [OK]

Hint: Freezing means no weight changes during training [OK]

Common Mistakes:

• Thinking freezing increases learning rate
• Believing freezing removes layers
• Assuming freezing duplicates layers

2. Which of the following is the correct way to freeze all parameters in a PyTorch model named model?

easy

A. model.freeze()

B. for param in model.parameters(): param.requires_grad = False

C. model.requires_grad = False

D. for param in model.parameters(): param.grad = None

Solution

1. Step 1: Identify correct syntax to freeze parameters

   Freezing requires setting requires_grad = False for each parameter.

2. Step 2: Check each option

   for param in model.parameters(): param.requires_grad = False correctly loops over parameters and sets requires_grad = False. Others are invalid or incorrect.

3. Final Answer:

   for param in model.parameters(): param.requires_grad = False -> Option B

4. Quick Check:

   Set requires_grad False to freeze [OK]

Hint: Set requires_grad=False on each parameter to freeze [OK]

Common Mistakes:

• Using model.requires_grad instead of param.requires_grad
• Calling a non-existent freeze() method
• Setting param.grad to None does not freeze

3. Consider this PyTorch code snippet:

import torch.nn as nn
model = nn.Sequential(
  nn.Linear(10, 5),
  nn.ReLU(),
  nn.Linear(5, 2)
)
for param in model[0].parameters():
  param.requires_grad = False

trainable_params = [p for p in model.parameters() if p.requires_grad]
print(len(trainable_params))

What will be printed?

medium

A. 2

B. 4

C. 6

D. 0

Solution

1. Step 1: Analyze model layers and parameters

   model[0] is Linear(10,5) with 2 parameters (weight and bias). model[2] is Linear(5,2) with 2 parameters.

2. Step 2: Check which parameters are trainable

   Parameters in model[0] are frozen (requires_grad=False), so only model[2]'s 2 parameters remain trainable.

3. Final Answer:

   2 -> Option A

4. Quick Check:

   Frozen layer params excluded, trainable = 2 [OK]

Hint: Count only parameters with requires_grad=True [OK]

Common Mistakes:

• Counting all parameters ignoring requires_grad
• Assuming ReLU has parameters
• Confusing layer indices

4. You want to freeze the first layer of a PyTorch model but accidentally wrote:

for param in model.layer1.parameters():
    param.grad = False

What is the problem with this code?

medium

A. param.grad is a tensor, not a boolean flag

B. param.grad disables gradients correctly

C. model.layer1.parameters() does not exist

D. param.requires_grad should be set, not param.grad

Solution

1. Step 1: Understand difference between param.grad and param.requires_grad

   param.grad holds gradient values, it is a tensor or None, not a flag to enable/disable gradients.

2. Step 2: Correct way to freeze parameters

   To freeze, set param.requires_grad = False. Setting param.grad = False is invalid and does not freeze.

3. Final Answer:

   param.requires_grad should be set, not param.grad -> Option D

4. Quick Check:

   Freeze by requires_grad=False, not param.grad [OK]

Hint: Freeze with requires_grad, not param.grad [OK]

Common Mistakes:

• Confusing param.grad with requires_grad
• Trying to disable gradients by setting param.grad
• Assuming param.grad is a boolean

5. You have a pretrained PyTorch model with 3 layers: layer1, layer2, and layer3. You want to freeze layer1 and layer2 but train layer3. Which code correctly freezes only the first two layers?

hard

A. for layer in [model.layer1, model.layer2]:
for param in layer.parameters():
param.requires_grad = False

B. for param in model.parameters():
param.requires_grad = False
for param in model.layer3.parameters():
param.requires_grad = False

C. model.layer1.requires_grad = False
model.layer2.requires_grad = False

D. model.freeze_layers(['layer1', 'layer2'])

Solution

1. Step 1: Understand freezing multiple layers

   Freezing means setting requires_grad = False on each parameter in the layers to freeze.

2. Step 2: Evaluate options

   for layer in [model.layer1, model.layer2]: for param in layer.parameters(): param.requires_grad = False correctly loops over layer1 and layer2 parameters and freezes them correctly. for param in model.parameters(): param.requires_grad = False for param in model.layer3.parameters(): param.requires_grad = False incorrectly freezes all parameters including layer3. model.layer1.requires_grad = False model.layer2.requires_grad = False tries to set requires_grad on layers (invalid). model.freeze_layers(['layer1', 'layer2']) calls a non-existent method.

3. Final Answer:

   for layer in [model.layer1, model.layer2]: for param in layer.parameters(): param.requires_grad = False -> Option A

4. Quick Check:

   Freeze layers by setting requires_grad False per parameter [OK]

Hint: Freeze layers by looping params and setting requires_grad False [OK]

Common Mistakes:

• Setting requires_grad on layer objects instead of parameters
• Using non-existent freeze_layers method
• Freezing all parameters