3D object detection in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - 3D object detection

Which metric matters for 3D object detection and WHY

In 3D object detection, we want to find objects in space accurately. The key metrics are Average Precision (AP) and Intersection over Union (IoU). AP tells us how well the model finds objects without many mistakes. IoU measures how much the predicted 3D box overlaps with the true box. A higher IoU means better localization. We also look at Recall to see if the model finds most objects, and Precision to check if the found objects are correct. These metrics help us understand both detection accuracy and location quality.

Confusion matrix for 3D object detection

3D object detection is more complex than simple classification, but we can think of detections as:

      +----------------+----------------+
      |                | Predicted Box  |
      |                | Present | None |
      +----------------+---------+------+
      | True Box       | TP      | FN   |
      | Present        |         |      |
      +----------------+---------+------+
      | True Box       | FP      | TN   |
      | Absent         |         |      |
      +----------------+---------+------+
    

Here, TP means the model correctly found a 3D box matching a real object with enough overlap (IoU above threshold). FP means the model found a box where no object exists. FN means the model missed a real object. TN is less common in detection but means correctly not detecting where no object is.

Precision vs Recall tradeoff with examples

Imagine a self-driving car detecting pedestrians in 3D space:

• High Precision, Low Recall: The car only signals pedestrians when very sure. Few false alarms, but it might miss some pedestrians. This is safer for avoiding false stops but risky if it misses people.
• High Recall, Low Precision: The car signals many possible pedestrians, catching almost all real ones but also many false alarms. This avoids missing anyone but may cause unnecessary stops.

We want a balance depending on the use case. For safety, high recall is often more important to avoid missing objects.

What "good" vs "bad" metric values look like for 3D object detection

Good 3D detection models typically have:

• Average Precision (AP): Above 70% is good; below 50% is poor.
• IoU Threshold: Usually 0.5 or 0.7; higher means stricter matching.
• Recall: Above 80% means most objects are found; below 50% means many misses.
• Precision: Above 80% means few false detections; below 50% means many false alarms.

Bad models might have low AP, low recall (missing objects), or low precision (many false boxes). Good models balance these well.

Common pitfalls in 3D object detection metrics

• Ignoring IoU thresholds: Reporting AP without a clear IoU cutoff can mislead about localization quality.
• Data leakage: Using test data in training inflates metrics falsely.
• Overfitting: Very high training AP but low test AP means the model memorizes training data, not generalizing.
• Class imbalance: Many background points but few objects can make accuracy look high but detection poor.
• Confusing precision and recall: Precision is about false alarms, recall about missed objects. Mixing them leads to wrong conclusions.

Self-check question

Your 3D object detection model has 98% accuracy but only 12% recall on detecting pedestrians. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most points are background (no pedestrians), so the model guesses "no pedestrian" most times correctly. But 12% recall means it misses 88% of pedestrians, which is dangerous for safety. High recall is critical to detect almost all pedestrians.

Key Result

Average Precision with IoU threshold and recall are key to evaluate 3D object detection quality.

Practice

1. What is the main goal of 3D object detection in computer vision?

easy

A. To classify images into categories

B. To find and locate objects in three-dimensional space

C. To enhance image colors

D. To compress video files

Solution

1. Step 1: Understand 3D object detection purpose

   3D object detection aims to find objects and their positions in 3D space, unlike simple image classification.

2. Step 2: Compare options to definition

   Only To find and locate objects in three-dimensional space describes locating objects in 3D space, which matches the goal of 3D object detection.

3. Final Answer:

   To find and locate objects in three-dimensional space -> Option B

4. Quick Check:

   3D object detection = locating objects in 3D space [OK]

Hint: 3D detection means finding objects in 3D space, not just classifying [OK]

Common Mistakes:

• Confusing 3D detection with image classification
• Thinking it changes image colors
• Assuming it compresses data

2. Which of the following is the correct way to represent a 3D bounding box in code?

easy

A. A 2D rectangle with width and height only

B. A single number representing volume

C. A color code string like '#FF0000'

D. A list of 8 corner points with (x, y, z) coordinates

Solution

1. Step 1: Recall 3D bounding box structure

   A 3D bounding box is defined by its 8 corners in 3D space, each with (x, y, z) coordinates.

2. Step 2: Evaluate options

   Only A list of 8 corner points with (x, y, z) coordinates correctly describes this. Options A, B, and D do not represent 3D bounding boxes properly.

3. Final Answer:

   A list of 8 corner points with (x, y, z) coordinates -> Option D

4. Quick Check:

   3D box = 8 corners with (x,y,z) [OK]

Hint: 3D boxes need 8 corners, not just volume or 2D shapes [OK]

Common Mistakes:

• Using only 2D rectangles for 3D boxes
• Confusing volume with box representation
• Using color codes instead of coordinates

3. Given the following Python code snippet for a simple 3D object detection model output, what will be the printed prediction?

predictions = {'car': [1.2, 3.4, 0.5], 'pedestrian': [2.1, 1.0, 0.3]}
print(predictions['car'])

medium

A. [1.2, 3.4, 0.5]

B. [2.1, 1.0, 0.3]

C. 'car'

D. KeyError

Solution

1. Step 1: Understand dictionary access in Python

   Accessing predictions['car'] returns the value associated with the key 'car', which is the list [1.2, 3.4, 0.5].

2. Step 2: Confirm output of print statement

   The print statement outputs the list [1.2, 3.4, 0.5], so [1.2, 3.4, 0.5] is correct.

3. Final Answer:

   [1.2, 3.4, 0.5] -> Option A

4. Quick Check:

   Dictionary access by key returns its value [OK]

Hint: Dictionary[key] returns the value for that key in Python [OK]

Common Mistakes:

• Confusing keys and values
• Expecting a KeyError without reason
• Printing the key instead of the value

4. The following code attempts to calculate the center of a 3D bounding box but has an error. What is the error?

def center_of_box(corners):
    x = (corners[0][0] + corners[1][0] + corners[2][0] + corners[3][0]) / 4
    y = (corners[0][1] + corners[1][1] + corners[2][1] + corners[3][1]) / 4
    z = (corners[0][2] + corners[1][2] + corners[2][2] + corners[3][2]) / 4
    return (x, y, z)

box_corners = [(1,2,3), (3,2,3), (3,4,3), (1,4,3), (1,2,5), (3,2,5), (3,4,5), (1,4,5)]
print(center_of_box(box_corners))

medium

A. The box_corners list has incorrect data types

B. The function uses wrong indices for coordinates

C. Only 4 corners are averaged instead of all 8

D. The function returns a list instead of a tuple

Solution

1. Step 1: Analyze the function's averaging method

   The function averages only the first 4 corners, ignoring the last 4 corners of the 3D box.

2. Step 2: Understand 3D box center calculation

   To find the true center, all 8 corners must be averaged, so the function misses half the points.

3. Final Answer:

   Only 4 corners are averaged instead of all 8 -> Option C

4. Quick Check:

   Center needs all 8 corners averaged [OK]

Hint: Average all 8 corners for center, not just 4 [OK]

Common Mistakes:

• Averaging only part of the corners
• Mixing up coordinate indices
• Confusing tuples and lists (not an error here)

5. In a 3D object detection system for self-driving cars, which metric best measures how well the predicted 3D bounding boxes match the true boxes?

hard

A. Intersection over Union (IoU) in 3D space

B. Pixel accuracy on 2D images

C. Mean Squared Error of RGB colors

D. Number of detected objects only

Solution

1. Step 1: Understand evaluation metrics for 3D detection

   IoU measures overlap between predicted and true boxes, extended to 3D for volume overlap.

2. Step 2: Compare other options

   Pixel accuracy and color errors do not measure 3D box quality; counting objects ignores box accuracy.

3. Final Answer:

   Intersection over Union (IoU) in 3D space -> Option A

4. Quick Check:

   3D IoU = best metric for 3D box accuracy [OK]

Hint: Use 3D IoU to measure box overlap accuracy [OK]

Common Mistakes:

• Using 2D pixel accuracy for 3D boxes
• Confusing color error with box accuracy
• Ignoring box overlap quality