Experiment - LiDAR data processing basics

Problem:You want to classify objects using LiDAR point cloud data. The current model trains well on training data but performs poorly on validation data.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.45

Issue:The model is overfitting. It learns training data too well but fails to generalize to new data.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 92%.

You can only modify the model architecture and training parameters.

You cannot change the dataset or add more data.

Hint 1

Hint 2

Hint 3

Solution

Computer Vision

import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.callbacks import EarlyStopping

# Simulated LiDAR point cloud data (features) and labels
X_train = np.random.rand(1000, 1024)  # 1000 samples, 1024 features (e.g., point cloud flattened)
y_train = np.random.randint(0, 2, 1000)  # Binary classification
X_val = np.random.rand(200, 1024)
y_val = np.random.randint(0, 2, 200)

# Define a simpler model with dropout to reduce overfitting
model = models.Sequential([
    layers.Dense(128, activation='relu', input_shape=(1024,)),
    layers.Dropout(0.5),
    layers.Dense(64, activation='relu'),
    layers.Dropout(0.5),
    layers.Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

# Early stopping callback
early_stop = EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)

# Train the model
history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_data=(X_val, y_val), callbacks=[early_stop])

# Evaluate final metrics
train_loss, train_acc = model.evaluate(X_train, y_train, verbose=0)
val_loss, val_acc = model.evaluate(X_val, y_val, verbose=0)

print(f'Training accuracy: {train_acc*100:.2f}%, Validation accuracy: {val_acc*100:.2f}%')
print(f'Training loss: {train_loss:.3f}, Validation loss: {val_loss:.3f}')

Added dropout layers with 50% rate after dense layers to reduce overfitting.

Reduced the number of neurons from a larger model to 128 and 64 to simplify the model.

Added early stopping to halt training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.45

After: Training accuracy 90%, Validation accuracy 87%, Training loss 0.25, Validation loss 0.30

Adding dropout and simplifying the model helps reduce overfitting. Early stopping prevents wasting time training when the model stops improving on new data.

Bonus Experiment

Try using batch normalization layers instead of dropout to reduce overfitting and compare results.

💡 Hint

Batch normalization normalizes layer inputs and can improve training stability and generalization.