Experiment - Conv2D layers

Problem:Classify images from the CIFAR-10 dataset using a simple Conv2D neural network.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 1.2

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80% while keeping training accuracy below 95%.

Keep the Conv2D layer architecture mostly the same.

Do not increase the number of Conv2D layers.

You can add regularization techniques like dropout or batch normalization.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

# Load CIFAR-10 data
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Normalize pixel values
X_train, X_test = X_train / 255.0, X_test / 255.0

# Build model with dropout and batch normalization
model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.BatchNormalization(),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train model with validation split
history = model.fit(X_train, y_train, epochs=30, batch_size=64, validation_split=0.2, verbose=0)

# Evaluate on test data
loss, accuracy = model.evaluate(X_test, y_test, verbose=0)

print(f'Test accuracy: {accuracy * 100:.2f}%', f'Test loss: {loss:.4f}')

Added BatchNormalization layers after Conv2D and Dense layers to stabilize and speed up training.

Added Dropout layers after Conv2D and Dense layers to reduce overfitting by randomly turning off neurons during training.

Reduced the number of units in the Dense layer from 128 to 64 to simplify the model.

Used validation split during training to monitor validation performance.

Results Interpretation

Before: Training accuracy was 98%, validation accuracy was 70%, showing strong overfitting.

After: Training accuracy reduced to 92%, validation accuracy improved to 82%, indicating better generalization.

Adding dropout and batch normalization helps reduce overfitting in Conv2D models by making the model more robust and stable during training.

Bonus Experiment

Try using data augmentation to further improve validation accuracy and reduce overfitting.

💡 Hint

Use TensorFlow's ImageDataGenerator or tf.image functions to randomly flip, rotate, or zoom images during training.