R-CNN family overview in Computer Vision - Model Metrics & Evaluation

Metrics & Evaluation - R-CNN family overview

Which metric matters for R-CNN family and WHY

For object detection models like the R-CNN family, the key metric is mean Average Precision (mAP). This metric measures how well the model finds and correctly labels objects in images. It balances both precision (how many detected objects are correct) and recall (how many true objects are found). We use mAP because object detection needs both accurate location and correct classification.

Confusion matrix for object detection (simplified)

       | Predicted Object | Predicted No Object
    -------------------------------------------
    True Object    |      TP          |        FN
    No Object      |      FP          |        TN
    -------------------------------------------
    Total samples = TP + FP + FN + TN
    

Here, TP means the model correctly found an object, FP means it found something that is not an object, FN means it missed an object, and TN means it correctly ignored background.

Precision vs Recall tradeoff with examples

In object detection:

• High precision means most detected objects are correct. This is important when false alarms are costly, like in self-driving cars where wrong detections can cause accidents.
• High recall means the model finds most of the true objects. This is important in security cameras where missing a person is risky.

Improving one often lowers the other. For example, setting a high confidence threshold increases precision but lowers recall.

What good vs bad metric values look like for R-CNN models

Good R-CNN model:

• mAP above 0.7 (70%) on standard datasets like COCO or PASCAL VOC
• Precision and recall balanced around 0.7 or higher
• Confusion matrix shows low FP and FN counts

Bad R-CNN model:

• mAP below 0.4 (40%) indicating poor detection or classification
• Very low recall (e.g., 0.2) means many objects missed
• Very low precision (e.g., 0.3) means many false detections

Common pitfalls in evaluating R-CNN models

• Accuracy paradox: High accuracy can be misleading if the dataset has many background images and few objects.
• Data leakage: Using test images during training inflates metrics falsely.
• Overfitting: Very high training mAP but low test mAP means the model memorizes training images and fails to generalize.
• Ignoring localization quality: Only counting classification accuracy without checking bounding box overlap (IoU) can give wrong impressions.

Self-check question

Your R-CNN model has 98% accuracy but only 12% recall on detecting pedestrians. Is it good for production? Why or why not?

Answer: No, it is not good. The high accuracy is misleading because most of the image is background (no pedestrians). The very low recall means the model misses most pedestrians, which is dangerous in real applications like self-driving cars or surveillance.

Key Result

Mean Average Precision (mAP) is the key metric for R-CNN models, balancing precision and recall to measure detection quality.

Practice

1. What is the main purpose of the R-CNN family of models in computer vision?

easy

A. To enhance image resolution

B. To generate captions for images

C. To detect edges in images

D. To find and classify objects in images by proposing regions

Solution

1. Step 1: Understand R-CNN model goal

   The R-CNN family is designed to locate objects by proposing regions and then classify them.

2. Step 2: Compare options with model purpose

   Only To find and classify objects in images by proposing regions matches the goal of finding and classifying objects using region proposals.

3. Final Answer:

   To find and classify objects in images by proposing regions -> Option D

4. Quick Check:

   R-CNN purpose = find and classify objects [OK]

Hint: R-CNN = Region proposals + Classification [OK]

Common Mistakes:

• Confusing R-CNN with image captioning models
• Thinking R-CNN enhances image quality
• Mixing edge detection with object detection

2. Which of the following is the correct sequence of steps in the original R-CNN pipeline?

easy

A. Classify entire image, extract regions, refine bounding boxes

B. Refine bounding boxes, extract regions, classify regions

C. Extract regions, classify regions, refine bounding boxes

D. Extract regions, refine bounding boxes, classify entire image

Solution

1. Step 1: Recall R-CNN pipeline steps

   R-CNN first extracts region proposals, then classifies each region, and finally refines bounding boxes.

2. Step 2: Match steps with options

   Extract regions, classify regions, refine bounding boxes correctly lists the steps in order: extract regions, classify, then refine boxes.

3. Final Answer:

   Extract regions, classify regions, refine bounding boxes -> Option C

4. Quick Check:

   R-CNN steps = extract, classify, refine [OK]

Hint: Remember: Propose regions first, then classify [OK]

Common Mistakes:

• Classifying before extracting regions
• Refining boxes before classification
• Mixing order of steps

3. Given the following code snippet for Faster R-CNN, what is the output shape of the feature map if the input image is 224x224 and the backbone reduces size by a factor of 16?

input_image = torch.randn(1, 3, 224, 224)
features = backbone(input_image)
print(features.shape)

medium

A. [1, C, 14, 14]

B. [1, C, 7, 7]

C. [1, C, 28, 28]

D. [1, C, 224, 224]

Solution

1. Step 1: Calculate spatial size after backbone

   The backbone reduces the input size by factor 16, so 224 / 16 = 14.

2. Step 2: Determine output shape format

   Output shape is batch size 1, channels C (depends on backbone), height 14, width 14.

3. Final Answer:

   [1, C, 14, 14] -> Option A

4. Quick Check:

   224/16 = 14, so output height and width = 14 [OK]

Hint: Divide input size by backbone stride for output size [OK]

Common Mistakes:

• Using 7 instead of 14 for output size
• Confusing channels with spatial dimensions
• Assuming no size reduction

4. You have a Mask R-CNN model but it is not producing masks for detected objects. Which of the following is the most likely cause?

medium

A. The backbone network is too shallow

B. The model was trained without the mask branch enabled

C. The input images are grayscale instead of RGB

D. The bounding box proposals are too large

Solution

1. Step 1: Understand Mask R-CNN mask output

   Mask R-CNN produces masks only if the mask branch is included and trained.

2. Step 2: Analyze options for mask absence

   If the mask branch is disabled during training, no masks will be produced, making The model was trained without the mask branch enabled correct.

3. Final Answer:

   The model was trained without the mask branch enabled -> Option B

4. Quick Check:

   Mask branch enabled = masks output [OK]

Hint: Masks need mask branch enabled and trained [OK]

Common Mistakes:

• Blaming backbone depth for missing masks
• Assuming grayscale input stops mask output
• Thinking bounding box size affects mask generation

5. You want to improve the speed of an R-CNN model without losing much accuracy. Which approach from the R-CNN family is best suited for this?

hard

A. Use Faster R-CNN with a Region Proposal Network (RPN)

B. Use Fast R-CNN with selective search region proposals

C. Use Mask R-CNN with added mask prediction

D. Use the original R-CNN with SVM classifiers

Solution

1. Step 1: Identify speed improvements in R-CNN family

   Faster R-CNN improves speed by integrating a Region Proposal Network to generate proposals efficiently.

2. Step 2: Compare options for speed and accuracy

   Fast R-CNN still uses slow selective search; Mask R-CNN adds mask overhead; original R-CNN is slow. So Faster R-CNN is best.

3. Final Answer:

   Use Faster R-CNN with a Region Proposal Network (RPN) -> Option A

4. Quick Check:

   Faster R-CNN = speed + accuracy via RPN [OK]

Hint: Faster R-CNN uses RPN for faster proposals [OK]

Common Mistakes:

• Choosing Fast R-CNN which still uses slow proposals
• Adding mask prediction which slows model
• Using original R-CNN which is slow