Freezing and unfreezing layers in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does it mean to 'freeze' layers in a neural network?

Freezing layers means making their weights unchangeable during training. This keeps the learned features intact while training other parts of the model.

beginner

Why would you freeze layers when using transfer learning?

You freeze layers to keep the useful features learned from a large dataset and only train new layers for your specific task. This saves time and avoids losing good features.

beginner

How do you freeze layers in TensorFlow Keras?

Set the layer's 'trainable' property to False, for example: layer.trainable = False. Then recompile and train the model.

intermediate

What does 'unfreezing' layers mean and when is it useful?

Unfreezing means allowing previously frozen layers to update their weights during training. It is useful for fine-tuning the whole model after initial training.

intermediate

What is a common workflow involving freezing and unfreezing layers?

First, freeze base layers and train new layers. Then unfreeze some or all base layers and continue training with a low learning rate to fine-tune.

What happens when you set layer.trainable = False in TensorFlow?

A The layer's weights do not update during training

B The layer's weights are reset to zero

C The layer is removed from the model

D The layer trains faster

Setting layer.trainable = False freezes the layer so its weights stay fixed during training.

Why freeze layers in transfer learning?

A To make the model smaller

B To speed up data loading

C To keep learned features and avoid retraining all layers

D To increase the number of layers

Freezing layers keeps the useful features learned from previous data and focuses training on new layers.

When is unfreezing layers most useful?

A Before compiling the model

B During fine-tuning after initial training

C When initializing the model

D When saving the model

Unfreezing allows updating weights to fine-tune the model after training new layers.

What must you do after changing layer.trainable before training?

A Recompile the model

B Restart Python

C Save the model

D Nothing, just train

You must recompile the model so the training configuration updates with the new trainable settings.

Which is a typical step in freezing and unfreezing workflow?

A Freeze all layers and never train

B Unfreeze all layers at the start

C Train only frozen layers

D Freeze base layers, train new layers, then unfreeze base layers for fine-tuning

This workflow helps keep learned features and then fine-tunes the whole model carefully.

Explain the concept of freezing and unfreezing layers in neural networks and why it is important in transfer learning.

Think about keeping learned features fixed vs. updating them.

Describe the steps to freeze and then unfreeze layers in a TensorFlow Keras model during training.

Remember to recompile after changing trainable status.

Practice

1. What does freezing layers in a TensorFlow model do?

easy

A. Stops the layers from updating their weights during training

B. Allows the layers to learn new features

C. Deletes the layers from the model

D. Increases the learning rate for those layers

Solution

1. Step 1: Understand the meaning of freezing layers

   Freezing layers means preventing them from changing their weights during training.

2. Step 2: Identify the effect of freezing

   Since weights do not update, the layers keep their learned knowledge intact.

3. Final Answer:

   Stops the layers from updating their weights during training -> Option A

4. Quick Check:

   Freezing = stop weight updates [OK]

Hint: Freezing means no weight updates during training [OK]

Common Mistakes:

• Thinking freezing deletes layers
• Believing freezing increases learning rate
• Confusing freezing with unfreezing

2. Which is the correct way to freeze a layer named layer in TensorFlow before compiling the model?

easy

A. layer.frozen = True

B. layer.freeze = True

C. layer.trainable = True

D. layer.trainable = False

Solution

1. Step 1: Recall the attribute to control training

   TensorFlow uses the trainable attribute to allow or prevent training on layers.

2. Step 2: Set the attribute to freeze

   Setting trainable = False freezes the layer so it won't update during training.

3. Final Answer:

   layer.trainable = False -> Option D

4. Quick Check:

   Freeze layer = trainable False [OK]

Hint: Use trainable = False to freeze layers before compile [OK]

Common Mistakes:

• Using non-existent attributes like freeze or frozen
• Setting trainable = True to freeze
• Changing trainable after compiling

3. Given the code below, what will be the output of model.layers[0].trainable after compiling?

model.layers[0].trainable = False
model.compile(optimizer='adam', loss='mse')
print(model.layers[0].trainable)

medium

A. None

B. True

C. False

D. Raises an error

Solution

1. Step 1: Set layer.trainable before compiling

   The code sets trainable = False before calling compile.

2. Step 2: Check if trainable attribute changes after compile

   The trainable attribute remains as set before compile; it does not change automatically.

3. Final Answer:

   False -> Option C

4. Quick Check:

   trainable stays False after compile [OK]

Hint: Set trainable before compile; it stays unchanged after [OK]

Common Mistakes:

• Assuming compile resets trainable to True
• Expecting an error from accessing trainable
• Confusing trainable with layer existence

4. You want to unfreeze a previously frozen layer in your TensorFlow model. Which code snippet fixes the error below?

AttributeError: 'Sequential' object has no attribute 'unfreeze'

Current code:

model.unfreeze()
model.compile(optimizer='adam', loss='mse')

medium

A. Set trainable on each layer: for layer in model.layers: layer.trainable = True

B. Set model.trainable = False instead

C. Call model.unfreeze() method

D. Rebuild the model from scratch

Solution

1. Step 1: Understand the error cause

   The error says Sequential model has no unfreeze attribute, meaning calling model.unfreeze() is invalid.

2. Step 2: Correct way to unfreeze layers

   Each layer has a trainable attribute; set it to True for all layers to unfreeze.

3. Final Answer:

   Set trainable on each layer: for layer in model.layers: layer.trainable = True -> Option A

4. Quick Check:

   Unfreeze layers by setting layer.trainable True [OK]

Hint: Set trainable on layers, not on model object [OK]

Common Mistakes:

• Trying to call model.unfreeze()
• Using non-existent unfreeze() method
• Rebuilding model unnecessarily

5. You have a pretrained model and want to freeze all layers except the last two for fine-tuning. Which code correctly achieves this before compiling?

hard

A. for layer in model.layers: layer.trainable = True

B. for layer in model.layers[:-2]: layer.trainable = False
for layer in model.layers[-2:]: layer.trainable = True

C. model.trainable = False
model.layers[-2:].trainable = True

D. model.freeze_layers(except_last=2)

Solution

1. Step 1: Freeze all layers except last two

   Use slicing to set trainable = False for all layers except last two.

2. Step 2: Unfreeze last two layers

   Set trainable = True for the last two layers to allow training.

3. Final Answer:

   for layer in model.layers[:-2]: layer.trainable = False for layer in model.layers[-2:]: layer.trainable = True -> Option B

4. Quick Check:

   Freeze all but last two layers [OK]

Hint: Freeze with slicing; unfreeze last layers explicitly [OK]

Common Mistakes:

• Trying to set trainable on model object
• Using non-existent freeze_layers method
• Setting all layers trainable True