DNN-based face detection in Computer Vision - ML Experiment: Train & Evaluate

Experiment - DNN-based face detection

Problem:Detect faces in images using a deep neural network (DNN). The current model is trained on a small dataset and achieves high training accuracy but low validation accuracy.

Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.45

Issue:The model is overfitting: it performs very well on training data but poorly on validation data.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training hyperparameters.

Do not change the dataset or add new data.

Keep the input image size and preprocessing steps the same.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models

# Define the improved model with dropout and batch normalization
model = models.Sequential([
    layers.Input(shape=(64, 64, 3)),
    layers.Conv2D(32, (3, 3), activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Conv2D(64, (3, 3), activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling2D((2, 2)),
    layers.Dropout(0.25),

    layers.Flatten(),
    layers.Dense(64, activation='relu'),
    layers.BatchNormalization(),
    layers.Dropout(0.5),
    layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Assume X_train, y_train, X_val, y_val are preloaded datasets
# Train the model with early stopping
early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

history = model.fit(
    X_train, y_train,
    epochs=50,
    batch_size=32,
    validation_data=(X_val, y_val),
    callbacks=[early_stop]
)

Added dropout layers after convolutional and dense layers to reduce overfitting.

Added batch normalization layers to stabilize and speed up training.

Reduced the number of neurons in the dense layer from 128 to 64 to simplify the model.

Lowered the learning rate to 0.0005 for smoother convergence.

Added early stopping to prevent over-training.

Results Interpretation

Before: Training accuracy 98%, Validation accuracy 70%, Training loss 0.05, Validation loss 0.45

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.20, Validation loss 0.30

Adding dropout and batch normalization, simplifying the model, and using early stopping helped reduce overfitting. This improved validation accuracy while keeping training accuracy reasonable, showing better generalization to new data.

Bonus Experiment

Try using data augmentation techniques to artificially increase the training data variety and see if validation accuracy improves further.

💡 Hint

Use image transformations like rotation, flipping, and zooming during training to help the model learn more robust features.