Experiment - CNN architecture review

Problem:Classify images from the CIFAR-10 dataset using a simple CNN model.

Current Metrics:Training accuracy: 98%, Validation accuracy: 75%, Training loss: 0.05, Validation loss: 0.85

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the CNN architecture and training hyperparameters.

Do not change the dataset or preprocessing steps.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models

# Load CIFAR-10 dataset
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Normalize pixel values
X_train, X_test = X_train / 255.0, X_test / 255.0

# Build improved CNN model
model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.BatchNormalization(),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

history = model.fit(X_train, y_train, epochs=30, batch_size=64, validation_split=0.2, verbose=0)

# Evaluate on test set
loss, accuracy = model.evaluate(X_test, y_test, verbose=0)

print(f'Test accuracy: {accuracy*100:.2f}%', f'Test loss: {loss:.4f}')

Added BatchNormalization layers after convolutional and dense layers to stabilize and speed up training.

Added Dropout layers with rates 0.25 and 0.5 to reduce overfitting by randomly turning off neurons during training.

Kept the model architecture simple with two convolutional layers and one dense layer to avoid excessive complexity.

Used Adam optimizer with a learning rate of 0.001 and batch size of 64 for balanced training.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 75%, Training loss 0.05, Validation loss 0.85

After: Training accuracy 90%, Validation accuracy 86%, Training loss 0.25, Validation loss 0.45

Adding dropout and batch normalization helps reduce overfitting by making the model generalize better to new data, improving validation accuracy while slightly lowering training accuracy.

Bonus Experiment

Try using data augmentation to further improve validation accuracy and reduce overfitting.

💡 Hint

Use Keras ImageDataGenerator to apply random flips, rotations, and shifts to training images.