Fine-tuning approach in TensorFlow - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to load a pre-trained model for fine-tuning.

TensorFlow

base_model = tf.keras.applications.MobileNetV2(input_shape=(224, 224, 3), include_top=False, weights=[1])

Drag options to blanks, or click blank then click option'

ANone

B'random'

C'imagenet'

D'cifar10'

Attempts:

3 left

💡 Hint

Common Mistakes

Using None will not load pre-trained weights.

Using 'random' or 'cifar10' are invalid for this argument.

The pre-trained weights from ImageNet are loaded by setting weights='imagenet'. This helps the model start with learned features.

2fill in blank

medium

Complete the code to freeze the base model layers during fine-tuning.

TensorFlow

base_model.[1] = False

Drag options to blanks, or click blank then click option'

Arequires_grad

Btrainable

Cfrozen

Dfrozen_layers

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'frozen' or 'frozen_layers' are not valid attributes.

Using 'requires_grad' is PyTorch syntax.

Setting base_model.trainable = False freezes the layers so they are not updated during training.

3fill in blank

hard

Fix the error in the code to add a global average pooling layer after the base model.

TensorFlow

x = base_model.output
x = tf.keras.layers.[1]()(x)

Drag options to blanks, or click blank then click option'

AGlobalAveragePooling2D

BMaxPooling2D

CAveragePooling2D

DGlobalMaxPooling2D

Attempts:

3 left

💡 Hint

Common Mistakes

Using MaxPooling layers changes the pooling method.

AveragePooling2D is not global and requires pool size.

GlobalAveragePooling2D reduces each feature map to a single value by averaging, which is common after convolutional base models.

4fill in blank

hard

Fill both blanks to compile the fine-tuned model with an optimizer and loss function.

TensorFlow

model.compile(optimizer=tf.keras.optimizers.[1](learning_rate=0.0001), loss='[2]', metrics=['accuracy'])

Drag options to blanks, or click blank then click option'

AAdam

BSGD

Ccategorical_crossentropy

Dbinary_crossentropy

Attempts:

3 left

💡 Hint

Common Mistakes

Using SGD optimizer may require different learning rate tuning.

Using 'binary_crossentropy' is for two-class problems.

Adam optimizer is popular for fine-tuning with a small learning rate. For multi-class classification, 'categorical_crossentropy' is used as loss.

5fill in blank

hard

Fill all three blanks to create a dictionary of training callbacks for early stopping and saving the best model.

TensorFlow

callbacks = [
    tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=[1]),
    tf.keras.callbacks.ModelCheckpoint(filepath='best_model.h5', save_best_only=[2], monitor='[3]')
]

Drag options to blanks, or click blank then click option'

A3

BTrue

Cval_loss

DFalse

Attempts:

3 left

💡 Hint

Common Mistakes

Setting patience too high or zero.

Using False for save_best_only saves all models.

Monitoring training loss instead of validation loss.

EarlyStopping patience is set to 3 epochs. ModelCheckpoint saves only the best model based on validation loss.

Practice

1. What is the main purpose of fine-tuning a pretrained TensorFlow model?

easy

A. To adapt the model to a new task by training some layers

B. To train a model from scratch with random weights

C. To increase the size of the dataset automatically

D. To reduce the number of layers in the model

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means adjusting a pretrained model to work well on a new task by training some of its layers.

2. Step 2: Compare options

   Only To adapt the model to a new task by training some layers correctly describes this process. Other options describe unrelated tasks.

3. Final Answer:

   To adapt the model to a new task by training some layers -> Option A

4. Quick Check:

   Fine-tuning = adapt pretrained model [OK]

Hint: Fine-tuning means training some layers on new data [OK]

Common Mistakes:

• Thinking fine-tuning means training from scratch
• Confusing fine-tuning with data augmentation
• Assuming fine-tuning reduces model size

2. Which TensorFlow code snippet correctly freezes the base model layers before fine-tuning?

easy

A. base_model.layers.freeze = True

B. base_model.trainable = False

C. base_model.freeze()

D. base_model.trainable = True

Solution

1. Step 1: Recall TensorFlow syntax for freezing layers

   In TensorFlow, setting trainable = False freezes the layers.

2. Step 2: Check options

   base_model.trainable = False uses correct syntax. Options A, B, and C are invalid methods/attributes or incorrect (e.g., trainable = True unfreezes layers).

3. Final Answer:

   base_model.trainable = False -> Option B

4. Quick Check:

   Freeze layers = trainable False [OK]

Hint: Freeze layers by setting trainable to False [OK]

Common Mistakes:

• Using incorrect method names like freeze()
• Setting trainable to True to freeze layers
• Trying to freeze layers by modifying layer attributes directly

3. What will be the output shape of the model after adding a new Dense layer with 10 units on top of a base model output shape (None, 1280)?

base_model = tf.keras.applications.MobileNetV2(input_shape=(224,224,3), include_top=False, pooling='avg')
new_output = tf.keras.layers.Dense(10, activation='softmax')(base_model.output)
model = tf.keras.Model(inputs=base_model.input, outputs=new_output)
print(model.output_shape)

medium

A. (None, 128)

B. (None, 224, 224, 3)

C. (None, 10)

D. (10, None)

Solution

1. Step 1: Understand base model output shape

   The base model outputs a tensor with shape (None, 1280) due to pooling='avg'.

2. Step 2: Adding Dense layer changes output shape

   The Dense layer with 10 units changes output shape to (None, 10), where None is batch size.

3. Final Answer:

   (None, 10) -> Option C

4. Quick Check:

   Dense units = output shape last dim [OK]

Hint: Dense layer units define output shape's last dimension [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch size dimension None
• Assuming output shape keeps base model shape

4. Identify the error in this fine-tuning code snippet:

base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_model.trainable = False

model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(5, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(train_data, epochs=5)

base_model.trainable = True
model.fit(train_data, epochs=5, learning_rate=0.001)

medium

A. Dense layer should be added before base_model

B. base_model.trainable should be True before first fit

C. Loss function is incorrect for classification

D. learning_rate is not a valid argument in model.fit

Solution

1. Step 1: Check model.fit arguments

   The model.fit method does not accept learning_rate as an argument; it belongs to the optimizer.

2. Step 2: Identify correct way to change learning rate

   To change learning rate, create a new optimizer with desired learning rate and recompile the model.

3. Final Answer:

   learning_rate is not a valid argument in model.fit -> Option D

4. Quick Check:

   learning_rate belongs to optimizer, not fit [OK]

Hint: learning_rate must be set in optimizer, not fit() [OK]

Common Mistakes:

• Passing learning_rate to model.fit
• Not recompiling model after changing trainable layers
• Freezing layers after first training instead of before

5. You want to fine-tune a pretrained model on a small dataset. Which approach is best to avoid overfitting?

# Step 1: Freeze base model
base_model.trainable = False
# Step 2: Add new classifier layers
# Step 3: Train new layers
# Step 4: Unfreeze some base layers
# Step 5: Continue training with low learning rate

Which of these is the best practice for Step 5?

hard

A. Use a very low learning rate like 1e-5 to fine-tune base layers

B. Use a high learning rate like 0.1 to speed up training

C. Keep learning rate same as initial training

D. Stop training immediately after unfreezing layers

Solution

1. Step 1: Understand fine-tuning with small data

   Fine-tuning with small data risks overfitting, so training must be gentle.

2. Step 2: Choose learning rate carefully

   A very low learning rate (e.g., 1e-5) helps adjust pretrained weights slowly to avoid overfitting.

3. Final Answer:

   Use a very low learning rate like 1e-5 to fine-tune base layers -> Option A

4. Quick Check:

   Low learning rate prevents overfitting in fine-tuning [OK]

Hint: Use low learning rate when unfreezing pretrained layers [OK]

Common Mistakes:

• Using high learning rate causing weight damage
• Not unfreezing any layers for fine-tuning
• Stopping training too early after unfreezing