Custom object detection dataset in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - Custom object detection dataset

Which metric matters for Custom Object Detection Dataset and WHY

For object detection, the key metric is mean Average Precision (mAP). It measures how well the model finds and correctly labels objects in images. mAP combines both precision (how many detected objects are correct) and recall (how many real objects are found). This balance is important because detecting too many wrong objects or missing real ones both hurt performance.

Confusion Matrix for Object Detection

Object detection uses a special confusion matrix based on True Positives (TP), False Positives (FP), and False Negatives (FN) per class. Here is a simple example for one class:

      +----------------+----------------+
      |                | Predicted Obj  |
      |                | Present | Absent |
      +----------------+---------+--------+
      | Actual Obj     |         |        |
      | Present       |   TP    |   FN   |
      | Absent        |   FP    |   TN*  |
      +----------------+---------+--------+

      *TN (True Negative) is less meaningful in object detection because background is large.
    

Precision = TP / (TP + FP)

Recall = TP / (TP + FN)

Precision vs Recall Tradeoff with Examples

In object detection:

• High Precision, Low Recall: The model finds fewer objects but is usually correct. Good when false alarms are costly, like detecting defects in products.
• High Recall, Low Precision: The model finds most objects but with many mistakes. Useful when missing any object is bad, like detecting pedestrians for safety.

Choosing the right balance depends on the task's needs.

What Good vs Bad Metric Values Look Like

For a custom object detection dataset:

• Good: mAP above 0.7 means the model detects most objects correctly with few mistakes.
• Bad: mAP below 0.3 means the model misses many objects or has many false detections.
• Precision and recall should both be reasonably high (above 0.6) for balanced performance.

Common Pitfalls in Metrics for Custom Object Detection

• Ignoring IoU Threshold: mAP depends on Intersection over Union (IoU) threshold. Too low threshold inflates scores; too high makes detection too strict.
• Data Leakage: Using test images in training leads to unrealistically high metrics.
• Class Imbalance: Rare classes may have low recall but get overshadowed in overall mAP.
• Overfitting: High training mAP but low test mAP means model memorizes training data, not generalizing.

Self Check

Your model has 98% accuracy but 12% recall on detecting a rare object. Is it good?

Answer: No. High accuracy can be misleading if most images don’t have the object. Low recall means the model misses most objects, which is bad for detection tasks.

Key Result

Mean Average Precision (mAP) is the key metric balancing precision and recall for custom object detection datasets.

Practice

1. What is the main purpose of creating a custom object detection dataset?

easy

A. To compress images for faster loading

B. To improve image resolution automatically

C. To generate random images for testing

D. To teach a computer to find specific objects in images

Solution

1. Step 1: Understand the goal of object detection datasets

   Custom object detection datasets are made to help computers learn to recognize and locate specific objects in images.

2. Step 2: Identify the correct purpose

   Among the options, only teaching a computer to find specific objects matches the goal of such datasets.

3. Final Answer:

   To teach a computer to find specific objects in images -> Option D

4. Quick Check:

   Custom dataset purpose = teach detection [OK]

Hint: Custom datasets train models to detect your objects [OK]

Common Mistakes:

• Confusing dataset purpose with image enhancement
• Thinking dataset creates images instead of labeling
• Assuming dataset compresses images

2. Which of the following is the correct format for an annotation line in a YOLO custom dataset label file?

easy

A. class_id x_center y_center width height (all normalized between 0 and 1)

B. image_width image_height class_id x y (pixel values)

C. class_name x_min y_min x_max y_max (absolute pixel coordinates)

D. class_id x_min y_min width height (normalized coordinates)

Solution

1. Step 1: Recall YOLO annotation format

   YOLO uses normalized coordinates with class_id, x_center, y_center, width, and height all between 0 and 1.

2. Step 2: Match options to YOLO format

   class_id x_center y_center width height (all normalized between 0 and 1) matches this format exactly; others use pixel values or different coordinate types.

3. Final Answer:

   class_id x_center y_center width height (all normalized between 0 and 1) -> Option A

4. Quick Check:

   YOLO labels use normalized center coords [OK]

Hint: YOLO labels use normalized center coordinates [OK]

Common Mistakes:

• Using pixel coordinates instead of normalized
• Confusing bounding box corners with center format
• Mixing class name with class id

3. Given this label file line for an image of size 640x480:
2 0.5 0.5 0.25 0.25
What are the pixel coordinates of the bounding box corners (x_min, y_min, x_max, y_max)?

medium

A. (200, 150, 360, 270)

B. (160, 120, 320, 240)

C. (240, 180, 400, 300)

D. (320, 240, 480, 360)

Solution

1. Step 1: Calculate center pixel coordinates

   x_center = 0.5 * 640 = 320, y_center = 0.5 * 480 = 240

2. Step 2: Calculate width and height in pixels

   width = 0.25 * 640 = 160, height = 0.25 * 480 = 120

3. Step 3: Calculate bounding box corners

   x_min = 320 - 160/2 = 240, y_min = 240 - 120/2 = 180
   x_max = 320 + 160/2 = 400, y_max = 240 + 120/2 = 300

4. Final Answer:

   (240, 180, 400, 300) -> Option C

5. Quick Check:

   Center ± half width/height = corners [OK]

Hint: Multiply normalized by image size, then subtract/add half box [OK]

Common Mistakes:

• Using width/height as full coordinates instead of half
• Mixing up x and y calculations
• Forgetting to multiply by image dimensions

4. You have a dataset folder with images and label files. The labels are missing for some images. What is the best way to fix this before training?

medium

A. Remove images without label files from the dataset

B. Create empty label files for those images

C. Rename images to match existing label files

D. Ignore missing labels and train anyway

Solution

1. Step 1: Understand dataset consistency needs

   Training requires each image to have a matching label file with annotations.

2. Step 2: Choose the best fix

   Removing images without labels avoids errors and ensures clean training data. Creating empty labels may confuse the model, renaming won't help if labels don't exist, ignoring missing labels causes errors.

3. Final Answer:

   Remove images without label files from the dataset -> Option A

4. Quick Check:

   Dataset must have matching image-label pairs [OK]

Hint: Keep only images with labels to avoid training errors [OK]

Common Mistakes:

• Creating empty label files that confuse training
• Ignoring missing labels causing crashes
• Renaming files without creating labels

5. You want to add a new object class to your existing custom dataset. Which steps should you follow to update your dataset correctly?

hard

A. Only add new images with the new class, no config changes needed

B. Add the new class to the class list, label new images with it, and update training config

C. Replace old class labels with the new class in all files

D. Add new class images but keep old labels unchanged without config update

Solution

1. Step 1: Update class list

   Add the new object class to your dataset's class list file so the model knows about it.

2. Step 2: Label new images with the new class

   Annotate images containing the new object using the new class id in label files.

3. Step 3: Update training configuration

   Modify your training config to include the new class count and class names.

4. Final Answer:

   Add the new class to the class list, label new images with it, and update training config -> Option B

5. Quick Check:

   Update classes, labels, and config for new objects [OK]

Hint: Update classes, label new data, and config for new objects [OK]

Common Mistakes:

• Not updating class list causing training errors
• Forgetting to label new objects
• Ignoring config updates leading to wrong class counts