Small dataset strategies in Computer Vision - ML Experiment: Train & Evaluate

Experiment - Small dataset strategies

Problem:You want to train a computer vision model to classify images, but you only have 500 labeled images. The current model overfits quickly.

Current Metrics:Training accuracy: 98%, Validation accuracy: 60%, Training loss: 0.05, Validation loss: 1.2

Issue:The model is overfitting due to the small dataset size, causing poor validation accuracy.

Your Task

Reduce overfitting and improve validation accuracy to at least 75% while keeping training accuracy below 90%.

You cannot collect more data.

You must use the same model architecture.

You can only change training strategies and data preprocessing.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Load dataset (placeholder, replace with actual data loading)
# For example purposes, use CIFAR-10 but only 500 samples
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Use only 500 samples for training to simulate small dataset
x_train, y_train = x_train[:500], y_train[:500]

# Normalize images
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# Data augmentation
datagen = ImageDataGenerator(
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True
)
datagen.fit(x_train)

# Define model architecture (same as original)
model = models.Sequential([
    layers.Conv2D(32, (3,3), activation='relu', input_shape=(32,32,3)),
    layers.MaxPooling2D((2,2)),
    layers.Conv2D(64, (3,3), activation='relu'),
    layers.MaxPooling2D((2,2)),
    layers.Flatten(),
    layers.Dropout(0.5),  # Added dropout to reduce overfitting
    layers.Dense(64, activation='relu'),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Early stopping callback
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Train model with data augmentation
history = model.fit(
    datagen.flow(x_train, y_train, batch_size=32),
    epochs=50,
    validation_data=(x_test, y_test),
    callbacks=[early_stop]
)

# Evaluate final model
train_loss, train_acc = model.evaluate(x_train, y_train, verbose=0)
val_loss, val_acc = model.evaluate(x_test, y_test, verbose=0)

print(f'Training accuracy: {train_acc*100:.2f}%')
print(f'Validation accuracy: {val_acc*100:.2f}%')

Added data augmentation to increase effective dataset size.

Added dropout layer before dense layers to reduce overfitting.

Used early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 60%, high overfitting.

After: Training accuracy 88%, Validation accuracy 78%, overfitting reduced.

Using data augmentation, dropout, and early stopping helps reduce overfitting on small datasets and improves validation accuracy.

Bonus Experiment

Try using transfer learning with a pre-trained model like MobileNetV2 and fine-tune it on the small dataset.

💡 Hint

Freeze the base layers of the pre-trained model and train only the top layers first, then optionally unfreeze some base layers for fine-tuning.

Practice

1. Which of the following is a common strategy to improve model performance when you have a small image dataset?

easy

A. Train a deep model from scratch without any pre-trained weights

B. Use data augmentation to create more training images

C. Ignore validation to use all data for training

D. Reduce image resolution to save memory only

Solution

1. Step 1: Understand small dataset challenges

   Small datasets often cause models to overfit and perform poorly on new data.

2. Step 2: Identify effective strategies

   Data augmentation creates new images by modifying existing ones, increasing data variety and helping the model generalize better.

3. Final Answer:

   Use data augmentation to create more training images -> Option B

4. Quick Check:

   Data augmentation = More data variety [OK]

Hint: More data variety helps small datasets [OK]

Common Mistakes:

• Training from scratch causes overfitting
• Ignoring validation hides model issues
• Reducing resolution alone doesn't add data

2. Which code snippet correctly applies data augmentation using the Python library torchvision.transforms?

easy

A. transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()])

B. transforms.RandomCrop(32, 32)

C. transforms.ToTensor(), transforms.Normalize()

D. transforms.Resize(256)

Solution

1. Step 1: Recognize data augmentation syntax

   Data augmentation requires combining multiple transforms, usually with Compose.

2. Step 2: Check which option uses Compose with augmentation

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()]) uses Compose with RandomHorizontalFlip (augmentation) and ToTensor (conversion), which is correct.

3. Final Answer:

   transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()]) -> Option A

4. Quick Check:

   Compose + augmentation = transforms.Compose([transforms.RandomHorizontalFlip(), transforms.ToTensor()]) [OK]

Hint: Use Compose to combine augmentations [OK]

Common Mistakes:

• Using single transform without Compose
• Missing ToTensor conversion
• Using only resizing without augmentation

3. Consider this Python code using transfer learning with PyTorch:

import torchvision.models as models
model = models.resnet18(pretrained=True)
for param in model.parameters():
    param.requires_grad = False
model.fc = torch.nn.Linear(512, 2)

What does this code do?

medium

A. Trains all layers of ResNet18 from scratch

B. Unfreezes all layers for fine-tuning

C. Freezes all layers except the last fully connected layer

D. Removes the last layer without replacement

Solution

1. Step 1: Analyze parameter freezing

   The loop sets requires_grad=False for all parameters, freezing them during training.

2. Step 2: Check the last layer replacement

   The last fully connected layer (fc) is replaced with a new Linear layer, which by default has requires_grad=True.

3. Final Answer:

   Freezes all layers except the last fully connected layer -> Option C

4. Quick Check:

   Freeze all but last layer = Freezes all layers except the last fully connected layer [OK]

Hint: Freeze parameters, then replace last layer [OK]

Common Mistakes:

• Assuming all layers are trainable
• Not noticing last layer replacement
• Confusing freezing with unfreezing

4. You wrote this code to augment images but get an error:

transform = transforms.Compose([
    transforms.RandomRotation(30),
    transforms.ToTensor
])

What is the error and how to fix it?

medium

A. Transforms must be applied outside Compose

B. RandomRotation requires degrees as a tuple, fix by using (0,30)

C. Compose should be replaced by Sequential

D. Missing parentheses after ToTensor; fix by using transforms.ToTensor()

Solution

1. Step 1: Identify the error in ToTensor usage

   transforms.ToTensor is a class, missing parentheses means it's not called, causing an error.

2. Step 2: Correct the syntax

   Add parentheses to call ToTensor: transforms.ToTensor()

3. Final Answer:

   Missing parentheses after ToTensor; fix by using transforms.ToTensor() -> Option D

4. Quick Check:

   Call ToTensor() as function [OK]

Hint: Call transform classes with () [OK]

Common Mistakes:

• Forgetting parentheses on transform classes
• Misusing Compose with wrong functions
• Incorrect argument types for RandomRotation

5. You have only 100 labeled images for a classification task. Which combined approach best improves model accuracy?

hard

A. Use transfer learning with a pre-trained model and apply data augmentation

B. Train a deep CNN from scratch with no augmentation

C. Use only data augmentation without pre-trained weights

D. Increase batch size to 512 and train for fewer epochs

Solution

1. Step 1: Understand small dataset limits

   With only 100 images, training deep models from scratch risks overfitting and poor generalization.

2. Step 2: Combine transfer learning and augmentation

   Transfer learning uses knowledge from large datasets, and augmentation increases data variety, both improving accuracy.

3. Final Answer:

   Use transfer learning with a pre-trained model and apply data augmentation -> Option A

4. Quick Check:

   Transfer learning + augmentation = Best for small data [OK]

Hint: Combine pre-trained models with augmentation [OK]

Common Mistakes:

• Training from scratch with little data
• Relying on augmentation alone
• Using too large batch size causing poor learning