Experiment - Benchmark datasets

Problem:You want to train a simple image classifier using a popular benchmark dataset like CIFAR-10. The current model achieves 95% training accuracy but only 70% validation accuracy.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.85

Issue:The model is overfitting the training data and does not generalize well to new images.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80% while keeping training accuracy below 90%.

Use the CIFAR-10 dataset only.

Keep the model architecture simple (e.g., a small CNN).

Do not increase the number of training epochs beyond 30.

Hint 1

Hint 2

Hint 3

Solution

Prompt Engineering / GenAI

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Load CIFAR-10 dataset
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Normalize pixel values
X_train, X_test = X_train / 255.0, X_test / 255.0

# Data augmentation
datagen = ImageDataGenerator(
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True
)
datagen.fit(X_train)

# Build a simple CNN model with dropout
model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),
    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train the model with data augmentation
history = model.fit(datagen.flow(X_train, y_train, batch_size=64),
                    epochs=30,
                    validation_data=(X_test, y_test),
                    verbose=2)

# Evaluate final model
final_train_loss, final_train_acc = model.evaluate(X_train, y_train, verbose=0)
final_val_loss, final_val_acc = model.evaluate(X_test, y_test, verbose=0)

print(f'Training accuracy: {final_train_acc*100:.2f}%')
print(f'Validation accuracy: {final_val_acc*100:.2f}%')

Added dropout layers after convolution and dense layers to reduce overfitting.

Applied data augmentation (rotation, shifts, flips) to increase training data variety.

Kept model architecture simple with two convolutional layers and one dense layer.

Used Adam optimizer with default learning rate and batch size of 64.

Limited training to 30 epochs.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.85

After: Training accuracy 88%, Validation accuracy 82%, Training loss 0.35, Validation loss 0.55

Adding dropout and data augmentation helps reduce overfitting, improving validation accuracy while slightly lowering training accuracy. This shows the importance of regularization and data variety in training models on benchmark datasets.

Bonus Experiment

Try using a different benchmark dataset like MNIST or Fashion-MNIST with the same model and techniques.

💡 Hint

Adjust input shape and number of classes accordingly, and observe if overfitting reduces similarly.