Fine-tuning approach in TensorFlow - Model Pipeline Trace

Model Pipeline - Fine-tuning approach

Fine-tuning is a way to teach a model by starting with a model already trained on a big dataset. We then adjust it a little bit with our own smaller dataset to make it work better for our specific task.

Data Flow - 5 Stages

1Load Pretrained Model

None→Load a model trained on a large dataset (e.g., ImageNet)→Model with fixed weights except last layers

A model trained to recognize 1000 types of objects

↓

2Replace Last Layer

Model output: 1000 classes→Replace last layer to match new task classes (e.g., 5 classes)→Model output: 5 classes

Change output layer to predict 5 types of flowers

↓

3Freeze Base Layers

Model layers→Freeze weights of base layers to keep learned features→Frozen base layers + trainable last layers

Base layers do not change during initial training

↓

4Train Last Layers

Training data: 500 images x 224x224x3→Train only last layers on new data→Updated last layers weights

Train last layers to recognize 5 flower types

↓

5Unfreeze and Fine-tune

Model with trained last layers→Unfreeze some base layers and train with low learning rate→Fine-tuned model weights

Adjust base layers slightly to improve accuracy

Training Trace - Epoch by Epoch

Loss
1.2 |************
0.9 |*********
0.7 |*******
0.6 |******
0.55|*****
0.52|****
0.48|****
0.45|***
0.43|**
0.42|*

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.55	Initial training of last layers, loss high, accuracy moderate
2	0.9	0.68	Loss decreases, accuracy improves as model learns
3	0.7	0.75	Continued improvement, model starts to generalize
4	0.6	0.80	Loss decreases steadily, accuracy rises
5	0.55	0.83	Last layers trained well, ready to unfreeze base layers
6	0.52	0.85	Unfreeze some base layers, start fine-tuning with low learning rate
7	0.48	0.87	Fine-tuning improves feature extraction, accuracy rises
8	0.45	0.89	Model converges with better accuracy and lower loss
9	0.43	0.90	Stable training, no overfitting observed
10	0.42	0.91	Final fine-tuned model ready for prediction

Prediction Trace - 6 Layers

Layer 1: Input Image

Layer 2: Base Layers (Frozen)

Layer 3: Fine-tuned Base Layers

Layer 4: New Last Layer

Layer 5: Softmax Activation

Layer 6: Prediction

Model Quiz - 3 Questions

Why do we freeze the base layers during initial training?

A To keep the useful features learned from the big dataset

B To make the model train faster by skipping layers

C Because base layers are not important for new tasks

D To prevent the model from making any predictions

Key Insight

Fine-tuning lets us reuse a big model's knowledge and adjust it gently to new tasks. This saves time and data, while improving accuracy by carefully training only parts of the model first, then fine-tuning more layers.

Practice

1. What is the main purpose of fine-tuning a pretrained TensorFlow model?

easy

A. To adapt the model to a new task by training some layers

B. To train a model from scratch with random weights

C. To increase the size of the dataset automatically

D. To reduce the number of layers in the model

Solution

1. Step 1: Understand fine-tuning concept

   Fine-tuning means adjusting a pretrained model to work well on a new task by training some of its layers.

2. Step 2: Compare options

   Only To adapt the model to a new task by training some layers correctly describes this process. Other options describe unrelated tasks.

3. Final Answer:

   To adapt the model to a new task by training some layers -> Option A

4. Quick Check:

   Fine-tuning = adapt pretrained model [OK]

Hint: Fine-tuning means training some layers on new data [OK]

Common Mistakes:

• Thinking fine-tuning means training from scratch
• Confusing fine-tuning with data augmentation
• Assuming fine-tuning reduces model size

2. Which TensorFlow code snippet correctly freezes the base model layers before fine-tuning?

easy

A. base_model.layers.freeze = True

B. base_model.trainable = False

C. base_model.freeze()

D. base_model.trainable = True

Solution

1. Step 1: Recall TensorFlow syntax for freezing layers

   In TensorFlow, setting trainable = False freezes the layers.

2. Step 2: Check options

   base_model.trainable = False uses correct syntax. Options A, B, and C are invalid methods/attributes or incorrect (e.g., trainable = True unfreezes layers).

3. Final Answer:

   base_model.trainable = False -> Option B

4. Quick Check:

   Freeze layers = trainable False [OK]

Hint: Freeze layers by setting trainable to False [OK]

Common Mistakes:

• Using incorrect method names like freeze()
• Setting trainable to True to freeze layers
• Trying to freeze layers by modifying layer attributes directly

3. What will be the output shape of the model after adding a new Dense layer with 10 units on top of a base model output shape (None, 1280)?

base_model = tf.keras.applications.MobileNetV2(input_shape=(224,224,3), include_top=False, pooling='avg')
new_output = tf.keras.layers.Dense(10, activation='softmax')(base_model.output)
model = tf.keras.Model(inputs=base_model.input, outputs=new_output)
print(model.output_shape)

medium

A. (None, 128)

B. (None, 224, 224, 3)

C. (None, 10)

D. (10, None)

Solution

1. Step 1: Understand base model output shape

   The base model outputs a tensor with shape (None, 1280) due to pooling='avg'.

2. Step 2: Adding Dense layer changes output shape

   The Dense layer with 10 units changes output shape to (None, 10), where None is batch size.

3. Final Answer:

   (None, 10) -> Option C

4. Quick Check:

   Dense units = output shape last dim [OK]

Hint: Dense layer units define output shape's last dimension [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch size dimension None
• Assuming output shape keeps base model shape

4. Identify the error in this fine-tuning code snippet:

base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_model.trainable = False

model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.Dense(5, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(train_data, epochs=5)

base_model.trainable = True
model.fit(train_data, epochs=5, learning_rate=0.001)

medium

A. Dense layer should be added before base_model

B. base_model.trainable should be True before first fit

C. Loss function is incorrect for classification

D. learning_rate is not a valid argument in model.fit

Solution

1. Step 1: Check model.fit arguments

   The model.fit method does not accept learning_rate as an argument; it belongs to the optimizer.

2. Step 2: Identify correct way to change learning rate

   To change learning rate, create a new optimizer with desired learning rate and recompile the model.

3. Final Answer:

   learning_rate is not a valid argument in model.fit -> Option D

4. Quick Check:

   learning_rate belongs to optimizer, not fit [OK]

Hint: learning_rate must be set in optimizer, not fit() [OK]

Common Mistakes:

• Passing learning_rate to model.fit
• Not recompiling model after changing trainable layers
• Freezing layers after first training instead of before

5. You want to fine-tune a pretrained model on a small dataset. Which approach is best to avoid overfitting?

# Step 1: Freeze base model
base_model.trainable = False
# Step 2: Add new classifier layers
# Step 3: Train new layers
# Step 4: Unfreeze some base layers
# Step 5: Continue training with low learning rate

Which of these is the best practice for Step 5?

hard

A. Use a very low learning rate like 1e-5 to fine-tune base layers

B. Use a high learning rate like 0.1 to speed up training

C. Keep learning rate same as initial training

D. Stop training immediately after unfreezing layers

Solution

1. Step 1: Understand fine-tuning with small data

   Fine-tuning with small data risks overfitting, so training must be gentle.

2. Step 2: Choose learning rate carefully

   A very low learning rate (e.g., 1e-5) helps adjust pretrained weights slowly to avoid overfitting.

3. Final Answer:

   Use a very low learning rate like 1e-5 to fine-tune base layers -> Option A

4. Quick Check:

   Low learning rate prevents overfitting in fine-tuning [OK]

Hint: Use low learning rate when unfreezing pretrained layers [OK]

Common Mistakes:

• Using high learning rate causing weight damage
• Not unfreezing any layers for fine-tuning
• Stopping training too early after unfreezing