Transfer learning for small datasets in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use transfer learning on small datasets?

Imagine you have a tiny dataset of 100 images to classify. Why is transfer learning a good choice here?

A It uses a pre-trained model to extract useful features, reducing the need for large data.

B It trains a new model from scratch to perfectly fit the small dataset.

C It ignores the small dataset and uses only the pre-trained model's predictions.

D It increases the dataset size by copying images multiple times.

💡 Hint

Think about how pre-trained models help when data is limited.

Transfer learning leverages knowledge from a large dataset to help learn features, so the small dataset doesn't need to teach everything from zero.

❓ Predict Output

intermediate

Output shape after freezing base model layers

Given this TensorFlow code snippet, what is the output shape of the model's predictions?

TensorFlow

import tensorflow as tf
base_model = tf.keras.applications.MobileNetV2(input_shape=(128,128,3), include_top=False, weights='imagenet')
base_model.trainable = False
model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(5, activation='softmax')
])
output_shape = model.output_shape

A (None, 128, 128, 3)

B (None, 5)

C (None, 7, 7, 1280)

D (None, 1)

💡 Hint

Check the last Dense layer's output units.

The model outputs a vector of length 5 for each input, matching the Dense layer's units.

❓ Hyperparameter

advanced

Best learning rate choice for fine-tuning a pre-trained model

You want to fine-tune a pre-trained model on a small dataset. Which learning rate is most suitable?

A 0.1 (high learning rate)

B 0.001 (moderate learning rate)

C 1.0 (extremely high learning rate)

D 0.00001 (very low learning rate)

💡 Hint

Fine-tuning usually requires careful, small updates.

A very low learning rate prevents large changes to the pre-trained weights, preserving learned features while adapting to new data.

❓ Metrics

advanced

Interpreting validation accuracy in transfer learning

After training a transfer learning model on a small dataset, you see training accuracy at 95% but validation accuracy at 60%. What does this indicate?

A The model is underfitting and needs more training.

B The validation data is too easy compared to training data.

C The model is overfitting the training data.

D The model is perfectly generalized.

💡 Hint

Think about the difference between training and validation accuracy.

High training accuracy but low validation accuracy means the model learned training data too well but fails to generalize.

🔧 Debug

expert

Why does this transfer learning code raise an error?

Consider this TensorFlow code snippet. It raises a ValueError when running. What is the cause?

TensorFlow

import tensorflow as tf
base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False, input_shape=(64,64,3))
base_model.trainable = False
model = tf.keras.Sequential([
    base_model,
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
import numpy as np
x = np.random.rand(32, 64, 64, 3)
y = np.random.randint(0, 10, size=(32,))
model.fit(x, y, epochs=1)

A Input shape (64,64,3) is too small for ResNet50 base model.

B The batch size 32 is too large for this model.

C The base model must be trainable for transfer learning to work.

D Loss function sparse_categorical_crossentropy is incompatible with softmax activation.

💡 Hint

Check the minimum input size requirements of ResNet50.

ResNet50 requires input images of at least 197x197 pixels when include_top=False; 64x64 is too small and causes a shape mismatch error.

Practice

1. What is the main reason to use transfer learning when working with small datasets in TensorFlow?

easy

A. To increase the size of the dataset automatically

B. To use a pretrained model's knowledge and avoid training from scratch

C. To reduce the number of layers in the model

D. To train all layers of a model from random weights

Solution

1. Step 1: Understand transfer learning purpose

   Transfer learning uses a pretrained model to leverage learned features from large datasets.

2. Step 2: Relate to small dataset challenges

   Small datasets lack enough data to train deep models well, so using pretrained knowledge helps.

3. Final Answer:

   To use a pretrained model's knowledge and avoid training from scratch -> Option B

4. Quick Check:

   Transfer learning = pretrained knowledge reuse [OK]

Hint: Transfer learning reuses pretrained features for small data [OK]

Common Mistakes:

• Thinking transfer learning increases dataset size
• Believing all layers must be retrained
• Assuming transfer learning reduces model layers

2. Which TensorFlow code snippet correctly freezes the base model layers for transfer learning?

easy

A. base_model.freeze_layers()

B. base_model.trainable = True

C. base_model.trainable = False

D. base_model.layers.freeze()

Solution

1. Step 1: Recall how to freeze layers in TensorFlow

   Setting trainable = False on the base model freezes all its layers.

2. Step 2: Check code options

   Only base_model.trainable = False is valid TensorFlow syntax to freeze layers.

3. Final Answer:

   base_model.trainable = False -> Option C

4. Quick Check:

   Freeze layers = trainable False [OK]

Hint: Set base_model.trainable = False to freeze layers [OK]

Common Mistakes:

• Using trainable = True instead of False
• Calling non-existent freeze_layers() method
• Trying to freeze layers with base_model.layers.freeze()

3. Given this code snippet, what will be the output shape of the model's predictions?

import tensorflow as tf
base_model = tf.keras.applications.MobileNetV2(input_shape=(128,128,3), include_top=False, pooling='avg')
base_model.trainable = False
model = tf.keras.Sequential([
  base_model,
  tf.keras.layers.Dense(10, activation='softmax')
])
model.build((None, 128, 128, 3))
predictions = model(tf.random.uniform((5, 128, 128, 3)))
print(predictions.shape)

medium

A. (5, 1)

B. (5, 128, 128, 3)

C. (5, 1000)

D. (5, 10)

Solution

1. Step 1: Analyze model architecture

   MobileNetV2 outputs pooled features (shape: (None, 1280)) due to pooling='avg'. Then Dense(10) outputs 10 classes.

2. Step 2: Check input batch size and output shape

   Input batch size is 5, so output shape is (5, 10) from Dense layer.

3. Final Answer:

   (5, 10) -> Option D

4. Quick Check:

   Batch 5, 10 classes output shape = (5, 10) [OK]

Hint: Output shape matches batch size and last Dense units [OK]

Common Mistakes:

• Confusing input shape with output shape
• Expecting output shape to match input image shape
• Assuming output has 1000 classes from original MobileNet

4. You wrote this code but get an error when training:

base_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
base_model.trainable = False
model = tf.keras.Sequential([
  base_model,
  tf.keras.layers.Dense(5, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(train_data, epochs=3)

What is the likely cause of the error?

medium

A. Missing input shape in base_model causing shape mismatch

B. trainable should be True to train the model

C. Loss function is incorrect for classification

D. Dense layer should have no activation

Solution

1. Step 1: Check base_model input shape requirement

   ResNet50 requires input_shape when include_top=False to build properly.

2. Step 2: Identify error cause

   Without input_shape, model layers don't connect correctly, causing training errors.

3. Final Answer:

   Missing input shape in base_model causing shape mismatch -> Option A

4. Quick Check:

   Input shape needed for base_model [OK]

Hint: Always specify input_shape when include_top=False [OK]

Common Mistakes:

• Not specifying input_shape for pretrained base model
• Thinking trainable must be True to avoid errors
• Changing loss function unnecessarily

5. You have a small dataset of 200 images for a 3-class classification task. Which transfer learning approach is best to get good accuracy quickly?

hard

A. Use a pretrained model, freeze base layers, add new Dense layers, train only new layers

B. Train a new model from scratch with random weights

C. Use a pretrained model and train all layers without freezing

D. Use a pretrained model without adding new layers and train only base layers

Solution

1. Step 1: Consider dataset size and training time

   Small dataset (200 images) benefits from freezing pretrained base to avoid overfitting and long training.

2. Step 2: Choose transfer learning strategy

   Freezing base layers and training new Dense layers adapts model to new task efficiently.

3. Final Answer:

   Use a pretrained model, freeze base layers, add new Dense layers, train only new layers -> Option A

4. Quick Check:

   Freeze base + train new layers = best for small data [OK]

Hint: Freeze base layers, train new layers for small datasets [OK]

Common Mistakes:

• Training all layers on small data causing overfitting
• Training from scratch wasting time and data
• Not adding new layers for the specific task