Image datasets (CIFAR-10, ImageNet) in Computer Vision - ML Experiment: Train & Evaluate

Experiment - Image datasets (CIFAR-10, ImageNet)

Problem:You are training a simple image classifier on the CIFAR-10 dataset. The model achieves 95% training accuracy but only 70% validation accuracy after 20 epochs.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.85

Issue:The model is overfitting: it performs very well on training data but poorly on validation data.

Your Task

Reduce overfitting so that validation accuracy improves to at least 80% while keeping training accuracy below 90%.

You can only modify the model architecture and training hyperparameters.

You cannot change the dataset or use additional data.

You must keep the model simple (no complex architectures).

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Load CIFAR-10 dataset
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()

# Normalize pixel values
X_train, X_test = X_train / 255.0, X_test / 255.0

# Data augmentation
datagen = ImageDataGenerator(
    rotation_range=15,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True
)
datagen.fit(X_train)

# Build a simple CNN model with dropout
model = models.Sequential([
    layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),
    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),
    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train the model with data augmentation
history = model.fit(datagen.flow(X_train, y_train, batch_size=64),
                    epochs=20,
                    validation_data=(X_test, y_test))

Added dropout layers after convolutional and dense layers to reduce overfitting.

Introduced data augmentation to increase training data variety.

Kept model architecture simple with two convolutional layers and one dense layer.

Used Adam optimizer with default learning rate and batch size of 64.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.85

After: Training accuracy 88%, Validation accuracy 82%, Training loss 0.35, Validation loss 0.55

Adding dropout and data augmentation helps reduce overfitting by making the model less confident on training data and more generalizable to new data.

Bonus Experiment

Try using a pretrained model like MobileNetV2 on CIFAR-10 and fine-tune it to improve validation accuracy further.

💡 Hint

Use transfer learning by freezing early layers and training only the last few layers with a low learning rate.

Practice

1. Which of the following best describes the CIFAR-10 dataset?

easy

A. A small dataset with 10 classes of images, easy for beginners

B. A very large dataset with millions of images and thousands of classes

C. A dataset mainly used for text recognition tasks

D. A dataset containing only black and white images

Solution

1. Step 1: Understand CIFAR-10 size and classes

   CIFAR-10 contains 60,000 small images divided into 10 classes, making it manageable for beginners.

2. Step 2: Compare with other datasets

   ImageNet is much larger with many more classes, unlike CIFAR-10.

3. Final Answer:

   A small dataset with 10 classes of images, easy for beginners -> Option A

4. Quick Check:

   CIFAR-10 = small, 10 classes [OK]

Hint: Remember CIFAR-10 is small and simple for learning [OK]

Common Mistakes:

• Confusing CIFAR-10 with ImageNet size
• Thinking CIFAR-10 has many classes
• Assuming CIFAR-10 is for text data

2. Which Python code correctly loads the CIFAR-10 dataset using TensorFlow?

easy

A. import cifar10
train_images, train_labels = cifar10.load()

B. from tensorflow.keras.datasets import cifar10
(train_images, train_labels), (test_images, test_labels) = cifar10.load_data()

C. from tensorflow.data import cifar10
train, test = cifar10.load()

D. from keras.datasets import imagenet
train, test = imagenet.load_data()

Solution

1. Step 1: Identify correct import for CIFAR-10 in TensorFlow

   The correct import is from tensorflow.keras.datasets import cifar10.

2. Step 2: Check the loading function

   cifar10.load_data() returns training and testing sets as tuples.

3. Final Answer:

   from tensorflow.keras.datasets import cifar10 (train_images, train_labels), (test_images, test_labels) = cifar10.load_data() -> Option B

4. Quick Check:

   Correct import and load_data() method [OK]

Hint: Use tensorflow.keras.datasets for CIFAR-10 loading [OK]

Common Mistakes:

• Using wrong module names like tensorflow.data
• Trying to load ImageNet with CIFAR-10 code
• Missing the load_data() function call

3. What will be the shape of the training images array after loading CIFAR-10 with TensorFlow?

medium

A. (100000, 64, 64, 3)

B. (60000, 28, 28, 1)

C. (50000, 32, 32, 3)

D. (50000, 224, 224, 3)

Solution

1. Step 1: Recall CIFAR-10 image count and size

   CIFAR-10 has 50,000 training images, each 32x32 pixels with 3 color channels (RGB).

2. Step 2: Match shape format

   The shape is (number_of_images, height, width, channels) = (50000, 32, 32, 3).

3. Final Answer:

   (50000, 32, 32, 3) -> Option C

4. Quick Check:

   Training images shape = (50000, 32, 32, 3) [OK]

Hint: CIFAR-10 images are 32x32 RGB, 50k training samples [OK]

Common Mistakes:

• Confusing CIFAR-10 with MNIST image size
• Using ImageNet image dimensions
• Mixing training and test set sizes

4. You wrote this code to load ImageNet but get an error:

from tensorflow.keras.datasets import imagenet
(train_images, train_labels), (test_images, test_labels) = imagenet.load_data()

What is the main problem?

medium

A. ImageNet is not available in tensorflow.keras.datasets module

B. The load_data() function requires extra parameters

C. You must import ImageNet from tensorflow.data instead

D. ImageNet images are grayscale, so loading fails

Solution

1. Step 1: Check TensorFlow dataset availability

   TensorFlow's keras.datasets does not include ImageNet; it includes CIFAR-10, MNIST, etc.

2. Step 2: Understand ImageNet loading method

   ImageNet requires special handling or external libraries, not keras.datasets.

3. Final Answer:

   ImageNet is not available in tensorflow.keras.datasets module -> Option A

4. Quick Check:

   ImageNet not in keras.datasets [OK]

Hint: ImageNet needs special loading, not keras.datasets [OK]

Common Mistakes:

• Assuming ImageNet loads like CIFAR-10
• Trying to import from wrong TensorFlow submodules
• Believing ImageNet images are grayscale

5. You want to train a model to recognize 1000 different object categories. Which dataset is best suited for this task?

hard

A. CIFAR-10, because it has 10 classes and is easy to use

B. Fashion-MNIST, because it has clothing images

C. MNIST, because it has handwritten digits

D. ImageNet, because it has 1000 classes and many images per class

Solution

1. Step 1: Identify dataset class count

   CIFAR-10 has only 10 classes, MNIST and Fashion-MNIST have 10 classes each, ImageNet has 1000 classes.

2. Step 2: Match dataset to task

   For recognizing 1000 categories, ImageNet is the suitable dataset due to its size and diversity.

3. Final Answer:

   ImageNet, because it has 1000 classes and many images per class -> Option D

4. Quick Check:

   1000 classes need ImageNet [OK]

Hint: Use ImageNet for many classes, CIFAR-10 for few [OK]

Common Mistakes:

• Choosing CIFAR-10 for many classes
• Confusing MNIST with ImageNet
• Ignoring class count importance