Freezing and unfreezing layers in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of model training with frozen layers

Consider a TensorFlow model where the first two layers are frozen before training. What will be the effect on the training loss after one epoch?

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

model = models.Sequential([
    layers.Dense(10, activation='relu', input_shape=(5,)),
    layers.Dense(10, activation='relu'),
    layers.Dense(1)
])

# Freeze first two layers
for layer in model.layers[:2]:
    layer.trainable = False

model.compile(optimizer='adam', loss='mse')

import numpy as np
x = np.random.random((100, 5))
y = np.random.random((100, 1))

history = model.fit(x, y, epochs=1, verbose=0)

print(round(history.history['loss'][0], 3))

A An error due to missing input shape specification

B A float value less than 1.0 representing the training loss

C A TypeError because frozen layers cannot be trained

D A float value greater than 10.0 representing the training loss

💡 Hint

Freezing layers means their weights do not update, but the model can still train other layers.

Freezing layers sets their trainable attribute to False, so their weights stay fixed. The model still trains the unfrozen layers, so training loss decreases and is a float value less than 1.0 after one epoch on random data.

❓ Model Choice

intermediate

Choosing layers to unfreeze for fine-tuning

You have a pretrained convolutional neural network for image classification. To fine-tune it on a new dataset, which layers should you unfreeze?

A No layers; keep all frozen

B All layers including the input layer

C Only the last few layers to adapt high-level features

D Only the first convolutional layer

💡 Hint

High-level features are learned in deeper layers.

Unfreezing only the last few layers allows the model to adapt to new data while keeping basic features intact. Unfreezing all layers may cause overfitting or require more data and time.

❓ Hyperparameter

advanced

Effect of freezing layers on learning rate choice

When you freeze most layers of a pretrained model and only train a few layers, how should you adjust the learning rate?

A Use a larger learning rate to speed up training

B Learning rate does not matter when layers are frozen

C Keep the learning rate the same as training from scratch

D Use a smaller learning rate to avoid large updates on few trainable layers

💡 Hint

Few trainable layers need careful updates to avoid destroying pretrained weights.

A smaller learning rate helps fine-tune the few trainable layers gently, preserving learned features and improving stability.

🔧 Debug

advanced

Why does unfreezing layers not update weights?

You unfreeze layers in a TensorFlow model by setting layer.trainable = True, but after training, weights do not change. What is the likely cause?

A The model was not recompiled after changing trainable flags

B The optimizer does not support training unfrozen layers

C The dataset is empty, so no training occurs

D TensorFlow does not allow unfreezing layers after compilation

💡 Hint

Changing trainable flags requires recompiling the model.

In TensorFlow, after changing layer.trainable, you must recompile the model to update the trainable variables list. Otherwise, training ignores the change.

🧠 Conceptual

expert

Impact of freezing layers on model capacity and generalization

Freezing layers in a pretrained model affects its capacity and generalization. Which statement best describes this impact?

A Freezing layers reduces model capacity but can improve generalization on small datasets

B Freezing layers increases model capacity and always improves generalization

C Freezing layers has no effect on capacity or generalization

D Freezing layers reduces generalization by limiting learning

💡 Hint

Think about how freezing limits learning but can prevent overfitting.

Freezing layers reduces the number of trainable parameters, lowering capacity. This can help generalization on small datasets by preventing overfitting.

Practice

1. What does freezing layers in a TensorFlow model do?

easy

A. Stops the layers from updating their weights during training

B. Allows the layers to learn new features

C. Deletes the layers from the model

D. Increases the learning rate for those layers

Solution

1. Step 1: Understand the meaning of freezing layers

   Freezing layers means preventing them from changing their weights during training.

2. Step 2: Identify the effect of freezing

   Since weights do not update, the layers keep their learned knowledge intact.

3. Final Answer:

   Stops the layers from updating their weights during training -> Option A

4. Quick Check:

   Freezing = stop weight updates [OK]

Hint: Freezing means no weight updates during training [OK]

Common Mistakes:

• Thinking freezing deletes layers
• Believing freezing increases learning rate
• Confusing freezing with unfreezing

2. Which is the correct way to freeze a layer named layer in TensorFlow before compiling the model?

easy

A. layer.frozen = True

B. layer.freeze = True

C. layer.trainable = True

D. layer.trainable = False

Solution

1. Step 1: Recall the attribute to control training

   TensorFlow uses the trainable attribute to allow or prevent training on layers.

2. Step 2: Set the attribute to freeze

   Setting trainable = False freezes the layer so it won't update during training.

3. Final Answer:

   layer.trainable = False -> Option D

4. Quick Check:

   Freeze layer = trainable False [OK]

Hint: Use trainable = False to freeze layers before compile [OK]

Common Mistakes:

• Using non-existent attributes like freeze or frozen
• Setting trainable = True to freeze
• Changing trainable after compiling

3. Given the code below, what will be the output of model.layers[0].trainable after compiling?

model.layers[0].trainable = False
model.compile(optimizer='adam', loss='mse')
print(model.layers[0].trainable)

medium

A. None

B. True

C. False

D. Raises an error

Solution

1. Step 1: Set layer.trainable before compiling

   The code sets trainable = False before calling compile.

2. Step 2: Check if trainable attribute changes after compile

   The trainable attribute remains as set before compile; it does not change automatically.

3. Final Answer:

   False -> Option C

4. Quick Check:

   trainable stays False after compile [OK]

Hint: Set trainable before compile; it stays unchanged after [OK]

Common Mistakes:

• Assuming compile resets trainable to True
• Expecting an error from accessing trainable
• Confusing trainable with layer existence

4. You want to unfreeze a previously frozen layer in your TensorFlow model. Which code snippet fixes the error below?

AttributeError: 'Sequential' object has no attribute 'unfreeze'

Current code:

model.unfreeze()
model.compile(optimizer='adam', loss='mse')

medium

A. Set trainable on each layer: for layer in model.layers: layer.trainable = True

B. Set model.trainable = False instead

C. Call model.unfreeze() method

D. Rebuild the model from scratch

Solution

1. Step 1: Understand the error cause

   The error says Sequential model has no unfreeze attribute, meaning calling model.unfreeze() is invalid.

2. Step 2: Correct way to unfreeze layers

   Each layer has a trainable attribute; set it to True for all layers to unfreeze.

3. Final Answer:

   Set trainable on each layer: for layer in model.layers: layer.trainable = True -> Option A

4. Quick Check:

   Unfreeze layers by setting layer.trainable True [OK]

Hint: Set trainable on layers, not on model object [OK]

Common Mistakes:

• Trying to call model.unfreeze()
• Using non-existent unfreeze() method
• Rebuilding model unnecessarily

5. You have a pretrained model and want to freeze all layers except the last two for fine-tuning. Which code correctly achieves this before compiling?

hard

A. for layer in model.layers: layer.trainable = True

B. for layer in model.layers[:-2]: layer.trainable = False
for layer in model.layers[-2:]: layer.trainable = True

C. model.trainable = False
model.layers[-2:].trainable = True

D. model.freeze_layers(except_last=2)

Solution

1. Step 1: Freeze all layers except last two

   Use slicing to set trainable = False for all layers except last two.

2. Step 2: Unfreeze last two layers

   Set trainable = True for the last two layers to allow training.

3. Final Answer:

   for layer in model.layers[:-2]: layer.trainable = False for layer in model.layers[-2:]: layer.trainable = True -> Option B

4. Quick Check:

   Freeze all but last two layers [OK]

Hint: Freeze with slicing; unfreeze last layers explicitly [OK]

Common Mistakes:

• Trying to set trainable on model object
• Using non-existent freeze_layers method
• Setting all layers trainable True