Experiment - SSD concept

Problem:You want to detect objects in images using the Single Shot MultiBox Detector (SSD) model. The current SSD model is trained on a small dataset but shows high training accuracy (95%) and low validation accuracy (60%).

Current Metrics:Training accuracy: 95%, Validation accuracy: 60%, Training loss: 0.15, Validation loss: 0.85

Issue:The model is overfitting: it performs very well on training data but poorly on validation data.

Your Task

Reduce overfitting so that validation accuracy improves to at least 75% while keeping training accuracy below 90%.

You can only modify the model architecture and training hyperparameters.

You cannot increase the dataset size or use external data.

You must keep the SSD model framework.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Define a simple SSD-like model with dropout
input_shape = (300, 300, 3)
num_classes = 21  # Example for VOC dataset

inputs = layers.Input(shape=input_shape)

# Base convolutional layers
x = layers.Conv2D(32, 3, activation='relu', padding='same')(inputs)
x = layers.MaxPooling2D(2)(x)
x = layers.Conv2D(64, 3, activation='relu', padding='same')(x)
x = layers.MaxPooling2D(2)(x)

# Add dropout to reduce overfitting
x = layers.Dropout(0.3)(x)

# Additional convolutional layers
x = layers.Conv2D(128, 3, activation='relu', padding='same')(x)
x = layers.MaxPooling2D(2)(x)
x = layers.Dropout(0.3)(x)

# Prediction layers (simplified for demonstration)
x = layers.Flatten()(x)
x = layers.Dense(256, activation='relu')(x)
x = layers.Dropout(0.4)(x)
outputs = layers.Dense(num_classes, activation='softmax')(x)

model = models.Model(inputs=inputs, outputs=outputs)

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='categorical_crossentropy',
              metrics=['accuracy'])

# Data augmentation setup
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    validation_split=0.2
)

# Assuming train_dir contains training images organized in class folders
train_generator = train_datagen.flow_from_directory(
    'train_dir',
    target_size=(300, 300),
    batch_size=32,
    class_mode='categorical',
    subset='training'
)

val_datagen = ImageDataGenerator(
    rescale=1./255,
    validation_split=0.2
)

validation_generator = val_datagen.flow_from_directory(
    'train_dir',
    target_size=(300, 300),
    batch_size=32,
    class_mode='categorical',
    subset='validation'
)

# Train the model with fewer epochs
history = model.fit(
    train_generator,
    epochs=15,
    validation_data=validation_generator
)

Added dropout layers after convolutional and dense layers to reduce overfitting.

Implemented data augmentation to increase data variety during training.

Lowered the learning rate from 0.001 to 0.0005 for smoother training.

Reduced the number of epochs from 50 to 15 to prevent memorization.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 60%, Training loss 0.15, Validation loss 0.85

After: Training accuracy 88%, Validation accuracy 78%, Training loss 0.30, Validation loss 0.55

Adding dropout and data augmentation helps the SSD model generalize better, reducing overfitting and improving validation accuracy.

Bonus Experiment

Try using batch normalization layers in the SSD model to improve training stability and further reduce overfitting.

💡 Hint

Insert batch normalization after convolutional layers before activation functions.