R-CNN family overview in Computer Vision - Model Pipeline Trace

Model Pipeline - R-CNN family overview

The R-CNN family is a set of models designed to find and classify objects in images. They work by first proposing regions that might contain objects, then extracting features from these regions, and finally classifying what object is inside each region.

Data Flow - 5 Stages

1Input Image

1 image x 600 x 800 x 3 channels→Original color image input→1 image x 600 x 800 x 3 channels

A photo of a street with cars and people

↓

2Region Proposal

1 image x 600 x 800 x 3 channels→Generate around 2000 candidate boxes (regions) that might contain objects→2000 regions x 4 coordinates each

Boxes around cars, people, signs in the image

↓

3Feature Extraction

2000 regions x variable size→Resize each region and extract features using a CNN→2000 regions x 4096 features

Feature vector representing each proposed region

↓

4Classification and Bounding Box Regression

2000 regions x 4096 features→Classify each region into object classes and refine box coordinates→2000 regions x (class scores + 4 refined coordinates)

Region classified as 'car' with adjusted box coordinates

↓

5Non-Maximum Suppression (NMS)

2000 regions x class scores and boxes→Remove overlapping boxes to keep best detections→Final detected objects, e.g., 10 boxes with classes

Final boxes around distinct cars and people without overlap

Training Trace - Epoch by Epoch

Loss
1.2 |*       
1.0 | *      
0.8 |  **    
0.6 |   **   
0.4 |    *** 
0.2 |      **
0.0 +--------
      1 5 10 15 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning to classify and localize objects
5	0.8	0.65	Loss decreases as model improves detection accuracy
10	0.5	0.78	Model shows good convergence with better localization
15	0.35	0.85	High accuracy and low loss indicate strong detection performance

Prediction Trace - 5 Layers

Layer 1: Input Image

Layer 2: Region Proposal

Layer 3: Feature Extraction CNN

Layer 4: Classification and Bounding Box Regression

Layer 5: Non-Maximum Suppression

Model Quiz - 3 Questions

What is the main role of the Region Proposal step in the R-CNN pipeline?

A Suggest areas in the image that might contain objects

B Classify objects inside the image

C Resize the input image

D Remove overlapping boxes

Key Insight

The R-CNN family breaks down object detection into clear steps: proposing regions, extracting features, classifying, and refining boxes. This modular approach helps the model learn to find and identify objects accurately by focusing on smaller parts of the image.

Practice

1. What is the main purpose of the R-CNN family of models in computer vision?

easy

A. To enhance image resolution

B. To generate captions for images

C. To detect edges in images

D. To find and classify objects in images by proposing regions

Solution

1. Step 1: Understand R-CNN model goal

   The R-CNN family is designed to locate objects by proposing regions and then classify them.

2. Step 2: Compare options with model purpose

   Only To find and classify objects in images by proposing regions matches the goal of finding and classifying objects using region proposals.

3. Final Answer:

   To find and classify objects in images by proposing regions -> Option D

4. Quick Check:

   R-CNN purpose = find and classify objects [OK]

Hint: R-CNN = Region proposals + Classification [OK]

Common Mistakes:

• Confusing R-CNN with image captioning models
• Thinking R-CNN enhances image quality
• Mixing edge detection with object detection

2. Which of the following is the correct sequence of steps in the original R-CNN pipeline?

easy

A. Classify entire image, extract regions, refine bounding boxes

B. Refine bounding boxes, extract regions, classify regions

C. Extract regions, classify regions, refine bounding boxes

D. Extract regions, refine bounding boxes, classify entire image

Solution

1. Step 1: Recall R-CNN pipeline steps

   R-CNN first extracts region proposals, then classifies each region, and finally refines bounding boxes.

2. Step 2: Match steps with options

   Extract regions, classify regions, refine bounding boxes correctly lists the steps in order: extract regions, classify, then refine boxes.

3. Final Answer:

   Extract regions, classify regions, refine bounding boxes -> Option C

4. Quick Check:

   R-CNN steps = extract, classify, refine [OK]

Hint: Remember: Propose regions first, then classify [OK]

Common Mistakes:

• Classifying before extracting regions
• Refining boxes before classification
• Mixing order of steps

3. Given the following code snippet for Faster R-CNN, what is the output shape of the feature map if the input image is 224x224 and the backbone reduces size by a factor of 16?

input_image = torch.randn(1, 3, 224, 224)
features = backbone(input_image)
print(features.shape)

medium

A. [1, C, 14, 14]

B. [1, C, 7, 7]

C. [1, C, 28, 28]

D. [1, C, 224, 224]

Solution

1. Step 1: Calculate spatial size after backbone

   The backbone reduces the input size by factor 16, so 224 / 16 = 14.

2. Step 2: Determine output shape format

   Output shape is batch size 1, channels C (depends on backbone), height 14, width 14.

3. Final Answer:

   [1, C, 14, 14] -> Option A

4. Quick Check:

   224/16 = 14, so output height and width = 14 [OK]

Hint: Divide input size by backbone stride for output size [OK]

Common Mistakes:

• Using 7 instead of 14 for output size
• Confusing channels with spatial dimensions
• Assuming no size reduction

4. You have a Mask R-CNN model but it is not producing masks for detected objects. Which of the following is the most likely cause?

medium

A. The backbone network is too shallow

B. The model was trained without the mask branch enabled

C. The input images are grayscale instead of RGB

D. The bounding box proposals are too large

Solution

1. Step 1: Understand Mask R-CNN mask output

   Mask R-CNN produces masks only if the mask branch is included and trained.

2. Step 2: Analyze options for mask absence

   If the mask branch is disabled during training, no masks will be produced, making The model was trained without the mask branch enabled correct.

3. Final Answer:

   The model was trained without the mask branch enabled -> Option B

4. Quick Check:

   Mask branch enabled = masks output [OK]

Hint: Masks need mask branch enabled and trained [OK]

Common Mistakes:

• Blaming backbone depth for missing masks
• Assuming grayscale input stops mask output
• Thinking bounding box size affects mask generation

5. You want to improve the speed of an R-CNN model without losing much accuracy. Which approach from the R-CNN family is best suited for this?

hard

A. Use Faster R-CNN with a Region Proposal Network (RPN)

B. Use Fast R-CNN with selective search region proposals

C. Use Mask R-CNN with added mask prediction

D. Use the original R-CNN with SVM classifiers

Solution

1. Step 1: Identify speed improvements in R-CNN family

   Faster R-CNN improves speed by integrating a Region Proposal Network to generate proposals efficiently.

2. Step 2: Compare options for speed and accuracy

   Fast R-CNN still uses slow selective search; Mask R-CNN adds mask overhead; original R-CNN is slow. So Faster R-CNN is best.

3. Final Answer:

   Use Faster R-CNN with a Region Proposal Network (RPN) -> Option A

4. Quick Check:

   Faster R-CNN = speed + accuracy via RPN [OK]

Hint: Faster R-CNN uses RPN for faster proposals [OK]

Common Mistakes:

• Choosing Fast R-CNN which still uses slow proposals
• Adding mask prediction which slows model
• Using original R-CNN which is slow