Conv2D layers in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Conv2D layers

Problem:Classify images from the CIFAR-10 dataset using a simple Conv2D neural network.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 1.2

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80% while keeping training accuracy below 95%.

Keep the Conv2D layer architecture mostly the same.

Do not increase the number of Conv2D layers.

You can add regularization techniques like dropout or batch normalization.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

# Load CIFAR-10 data
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Normalize pixel values
X_train, X_test = X_train / 255.0, X_test / 255.0

# Build model with dropout and batch normalization
model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.BatchNormalization(),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train model with validation split
history = model.fit(X_train, y_train, epochs=30, batch_size=64, validation_split=0.2, verbose=0)

# Evaluate on test data
loss, accuracy = model.evaluate(X_test, y_test, verbose=0)

print(f'Test accuracy: {accuracy * 100:.2f}%', f'Test loss: {loss:.4f}')

Added BatchNormalization layers after Conv2D and Dense layers to stabilize and speed up training.

Added Dropout layers after Conv2D and Dense layers to reduce overfitting by randomly turning off neurons during training.

Reduced the number of units in the Dense layer from 128 to 64 to simplify the model.

Used validation split during training to monitor validation performance.

Results Interpretation

Before: Training accuracy was 98%, validation accuracy was 70%, showing strong overfitting.

After: Training accuracy reduced to 92%, validation accuracy improved to 82%, indicating better generalization.

Adding dropout and batch normalization helps reduce overfitting in Conv2D models by making the model more robust and stable during training.

Bonus Experiment

Try using data augmentation to further improve validation accuracy and reduce overfitting.

💡 Hint

Use TensorFlow's ImageDataGenerator or tf.image functions to randomly flip, rotate, or zoom images during training.

Practice

1. What does a Conv2D layer primarily do in a neural network?

easy

A. It scans images with small windows to find features.

B. It reduces the size of the input image by half.

C. It converts images into text descriptions.

D. It sorts images based on color.

Solution

1. Step 1: Understand the role of Conv2D

   A Conv2D layer uses small filters to scan over images and detect patterns like edges or textures.

2. Step 2: Compare options with Conv2D function

   Only It scans images with small windows to find features. correctly describes this scanning and feature detection process.

3. Final Answer:

   It scans images with small windows to find features. -> Option A

4. Quick Check:

   Conv2D = feature scanning [OK]

Hint: Conv2D finds image features using small filters [OK]

Common Mistakes:

• Thinking Conv2D reduces image size directly
• Confusing Conv2D with image captioning
• Assuming Conv2D sorts images

2. Which of the following is the correct way to create a Conv2D layer with 32 filters and a 3x3 kernel in TensorFlow?

easy

A. tf.keras.layers.Conv2D(32, kernel=3)

B. tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3))

C. tf.keras.layers.Conv2D(filters=3, kernel_size=32)

D. tf.keras.layers.Conv2D(32, size=(3,3))

Solution

1. Step 1: Recall Conv2D syntax

   The correct syntax uses filters for number of filters and kernel_size as a tuple for filter size.

2. Step 2: Check each option

   tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3)) matches the correct syntax. Options B, C, and D use wrong parameter names or values.

3. Final Answer:

   tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3)) -> Option B

4. Quick Check:

   Correct parameters = tf.keras.layers.Conv2D(filters=32, kernel_size=(3,3)) [OK]

Hint: Use filters= and kernel_size=(h,w) in Conv2D [OK]

Common Mistakes:

• Using wrong parameter names like kernel or size
• Swapping filters and kernel_size values
• Passing kernel_size as a single integer without tuple

3. What will be the output shape of this Conv2D layer in TensorFlow?

layer = tf.keras.layers.Conv2D(filters=16, kernel_size=(3,3), padding='valid')
input_shape = (None, 28, 28, 1)
output_shape = layer.compute_output_shape(input_shape)

medium

A. (None, 30, 30, 16)

B. (None, 28, 28, 16)

C. (None, 26, 26, 16)

D. (None, 25, 25, 16)

Solution

1. Step 1: Understand padding='valid'

   Valid padding means no padding, so output size shrinks by kernel_size - 1 in height and width.

2. Step 2: Calculate output dimensions

   Input is 28x28, kernel is 3x3, so output height and width = 28 - 3 + 1 = 26. Filters = 16.

3. Final Answer:

   (None, 26, 26, 16) -> Option C

4. Quick Check:

   Output size = input - kernel + 1 [OK]

Hint: Valid padding shrinks output by kernel_size - 1 [OK]

Common Mistakes:

• Assuming output size stays same with valid padding
• Confusing padding='same' with 'valid'
• Forgetting to apply kernel size reduction

4. Identify the error in this Conv2D layer code:

model = tf.keras.Sequential([
  tf.keras.layers.Conv2D(64, kernel_size=3, input_shape=(28,28)),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(10, activation='softmax')
])

medium

A. Missing channel dimension in input_shape

B. kernel_size should be a tuple, not an integer

C. Dense layer should come before Flatten

D. Activation function 'softmax' is invalid here

Solution

1. Step 1: Check input_shape format

   Conv2D expects input_shape with 3 dimensions: height, width, and channels. Here channels are missing.

2. Step 2: Validate other parts

   kernel_size=3 is valid (interpreted as (3,3)), Dense after Flatten is correct, softmax is valid for classification.

3. Final Answer:

   Missing channel dimension in input_shape -> Option A

4. Quick Check:

   Input shape must include channels [OK]

Hint: Input shape for Conv2D needs channels (e.g., (28,28,1)) [OK]

Common Mistakes:

• Ignoring channel dimension in input_shape
• Thinking kernel_size must be tuple always
• Misordering Flatten and Dense layers

5. You want to build a Conv2D model that keeps the input image size unchanged after convolution. Which setting should you use for the padding parameter?

hard

A. 'valid' to avoid padding

B. 'none' to disable convolution

C. 'full' to increase output size

D. 'same' to keep output size equal to input size

Solution

1. Step 1: Understand padding options

   'valid' means no padding, output shrinks; 'same' adds padding to keep output size same as input.

2. Step 2: Choose padding to keep size

   To keep input and output size equal, use padding='same'. Options 'full' and 'none' are invalid in TensorFlow Conv2D.

3. Final Answer:

   'same' to keep output size equal to input size -> Option D

4. Quick Check:

   Padding='same' keeps size [OK]

Hint: Use padding='same' to keep image size after Conv2D [OK]

Common Mistakes:

• Choosing 'valid' and expecting same size
• Using unsupported padding values like 'full' or 'none'
• Confusing padding effect on output size