Transfer learning for small datasets in TensorFlow - Model Pipeline Trace

Model Pipeline - Transfer learning for small datasets

This pipeline uses a pre-trained model to learn from a small dataset quickly and accurately. It reuses learned features from a large dataset and fine-tunes the model on new data.

Data Flow - 5 Stages

1Load small dataset

100 rows x 224 x 224 x 3→Load images and labels for 100 samples→100 rows x 224 x 224 x 3

Image of a cat with label 'cat'

↓

2Preprocessing

100 rows x 224 x 224 x 3→Resize and normalize pixel values to [0,1]→100 rows x 224 x 224 x 3

Normalized image pixel values between 0 and 1

↓

3Load pre-trained base model

224 x 224 x 3→Use MobileNetV2 without top layers, freeze weights→100 rows x 7 x 7 x 1280

Feature map tensor for each image

↓

4Add new classification head

100 rows x 7 x 7 x 1280→Global average pooling + dense layer for 5 classes→100 rows x 5

Logits for 5 classes per image

↓

5Train on small dataset

100 rows x 5→Fine-tune classification head with frozen base→Model weights updated for classification head

Model learns to classify new images

Training Trace - Epoch by Epoch

Loss
1.2 |*       
1.0 | *      
0.8 |  *     
0.6 |   *    
0.4 |    *   
    +---------
     1 2 3 4 5
     Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.20	0.45	Model starts learning from small dataset
2	0.85	0.65	Loss decreases, accuracy improves
3	0.60	0.78	Model fine-tunes classification head well
4	0.50	0.82	Training stabilizes with good accuracy
5	0.45	0.85	Final epoch shows best performance

Prediction Trace - 5 Layers

Layer 1: Input image

Layer 2: Pre-trained base model (MobileNetV2)

Layer 3: Global average pooling

Layer 4: Dense classification layer

Layer 5: Softmax activation

Model Quiz - 3 Questions

Why do we freeze the base model weights during training?

A To increase the number of trainable parameters

B To keep learned features and avoid overfitting on small data

C To speed up training by skipping the classification head

D To prevent the model from making predictions

Key Insight

Transfer learning lets us use a big model trained on lots of data to quickly learn new tasks with small datasets. Freezing the base model keeps useful features, while training a small new head adapts to new classes efficiently.