Fine-tuning approach in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the right layer to fine-tune

You have a pre-trained convolutional neural network for image classification. You want to fine-tune it on a new dataset with similar images but different classes. Which layer is best to fine-tune to adapt the model effectively?

A The last few convolutional layers and the fully connected layers

B All layers including the input layer

C Only the first convolutional layer

D Only the last fully connected (dense) layer

💡 Hint

Think about which layers capture general features versus task-specific features.

The first layers learn general features like edges, which are useful across tasks. The last layers learn task-specific features. Fine-tuning the last few convolutional layers and fully connected layers allows adapting to new classes while keeping general features intact.

❓ Hyperparameter

intermediate

Selecting learning rate for fine-tuning

When fine-tuning a pre-trained model, which learning rate setting is generally recommended to avoid destroying the learned features?

A Use a very high learning rate like 0.1

B Use no learning rate, keep weights frozen

C Use a smaller learning rate than training from scratch, like 0.0001

D Use the same learning rate as training from scratch

💡 Hint

Consider how big changes to weights affect pre-trained knowledge.

A smaller learning rate helps make gentle updates to the pre-trained weights, preserving useful features while adapting to new data. A high learning rate can overwrite learned features too quickly.

❓ Predict Output

advanced

Output of fine-tuning code snippet

What will be the printed output after running this TensorFlow fine-tuning code snippet?

TensorFlow

import tensorflow as tf
from tensorflow.keras.applications import MobileNetV2

base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(96,96,3))
base_model.trainable = False

model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

print(f"Trainable layers before fine-tuning: {sum([layer.trainable for layer in base_model.layers])}")

base_model.trainable = True

print(f"Trainable layers after setting trainable=True: {sum([layer.trainable for layer in base_model.layers])}")

A

Trainable layers before fine-tuning: 0
Trainable layers after setting trainable=True: 0

B

Trainable layers before fine-tuning: 0
Trainable layers after setting trainable=True: 154

C

Trainable layers before fine-tuning: 154
Trainable layers after setting trainable=True: 0

D

Trainable layers before fine-tuning: 154
Trainable layers after setting trainable=True: 154

💡 Hint

Check the trainable attribute before and after changing it.

Initially, base_model.trainable is set to False, so no layers are trainable (0). After setting base_model.trainable = True, all layers become trainable. MobileNetV2 has 154 layers, so the count is 154.

❓ Metrics

advanced

Interpreting fine-tuning training metrics

You fine-tune a pre-trained model on a small dataset. After 10 epochs, training accuracy is 98% but validation accuracy is 70%. What does this indicate?

A The model has a bug in the code

B The model is perfectly generalized

C The model is underfitting the training data

D The model is overfitting the training data

💡 Hint

Compare training and validation accuracy to assess model behavior.

High training accuracy but much lower validation accuracy means the model learned the training data too well but does not generalize to new data, which is overfitting.

🔧 Debug

expert

Debugging fine-tuning freezing layers issue

You want to fine-tune only the last 10 layers of a pre-trained model in TensorFlow. You set base_model.trainable = True and then set the first layers' trainable attribute to False in a loop. However, after training, you notice all layers are still being updated. What is the likely cause?

TensorFlow

base_model.trainable = True
for layer in base_model.layers[:-10]:
    layer.trainable = False

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(train_data, epochs=5)

A You must compile the model after changing layer trainable attributes

B Setting base_model.trainable = True overrides individual layer settings

C You need to set model.trainable = False instead

D The optimizer must be reset to apply layer freezing

💡 Hint

Think about when TensorFlow reads layer trainable flags.

In TensorFlow, changing layer trainable flags after compiling the model has no effect until you recompile. The optimizer and trainable variables are fixed at compile time.