torchvision detection models in PyTorch - Model Pipeline Trace

Model Pipeline - torchvision detection models

This pipeline uses torchvision detection models to find and classify objects in images. It takes images, processes them, trains a detection model, and then predicts bounding boxes and labels for objects.

Data Flow - 7 Stages

1Input Images

1000 images x 3 channels x 224 height x 224 width→Raw images loaded from dataset→1000 images x 3 channels x 224 height x 224 width

Image of a dog with size 224x224 pixels

↓

2Preprocessing

1000 images x 3 channels x 224 height x 224 width→Normalize pixel values and convert to tensor→1000 images x 3 channels x 224 height x 224 width (tensor)

Normalized tensor representing the dog image

↓

3Feature Extraction

1000 images x 3 channels x 224 height x 224 width→Backbone CNN extracts features→1000 images x 256 channels x 7 height x 7 width

Feature map tensor highlighting edges and shapes

↓

4Region Proposal Network (RPN)

1000 images x 256 channels x 7 height x 7 width→Generate candidate object regions→1000 images x 300 proposals x 4 coordinates

Bounding box proposals like [x1, y1, x2, y2]

↓

5RoI Pooling

1000 images x 300 proposals x 256 channels x 7 x 7→Extract fixed-size feature maps for each proposal→1000 images x 300 proposals x 256 channels x 7 x 7

Feature maps cropped for each proposed region

↓

6Classification and Box Regression

1000 images x 300 proposals x 256 channels x 7 x 7→Predict class scores and refine bounding boxes→1000 images x 300 proposals x (num_classes + 4)

Class scores like 'dog: 0.9', bounding box coords refined

↓

7Post-processing

1000 images x 300 proposals x (num_classes + 4)→Apply Non-Maximum Suppression to remove overlaps→1000 images x variable proposals x (class + box)

Final boxes and labels like 'dog at [50,60,150,160]'

Training Trace - Epoch by Epoch

Epoch 1: ************ (1.2)
Epoch 2: ******** (0.9)
Epoch 3: ****** (0.7)
Epoch 4: **** (0.55)
Epoch 5: *** (0.45)

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Model starts learning, loss high, accuracy low
2	0.9	0.60	Loss decreases, accuracy improves
3	0.7	0.72	Model learns better features, accuracy rises
4	0.55	0.80	Loss continues to drop, accuracy nearing good levels
5	0.45	0.85	Training converging, model performs well

Prediction Trace - 6 Layers

Layer 1: Input Image

Layer 2: Backbone CNN

Layer 3: Region Proposal Network

Layer 4: RoI Pooling

Layer 5: Classification and Box Regression

Layer 6: Non-Maximum Suppression

Model Quiz - 3 Questions

What does the Region Proposal Network (RPN) do in the pipeline?

A Normalizes the input images

B Suggests candidate regions where objects might be

C Classifies the objects in the image

D Removes overlapping bounding boxes

Key Insight

Torchvision detection models use a step-by-step process: first extracting features, then proposing regions, classifying objects, and finally refining predictions. Training shows steady loss decrease and accuracy increase, indicating the model learns to detect objects well.

Practice

1. Which of the following is a detection model provided by torchvision for finding objects in images?

easy

A. ResNet

B. Faster R-CNN

C. VGG16

D. Transformer

Solution

1. Step 1: Understand torchvision detection models

   torchvision offers models specifically for object detection like Faster R-CNN, Mask R-CNN, and RetinaNet.

2. Step 2: Identify the correct model

   ResNet and VGG16 are classification models, and Transformer is a general architecture, not a torchvision detection model.

3. Final Answer:

   Faster R-CNN -> Option B

4. Quick Check:

   Detection model = Faster R-CNN [OK]

Hint: Detection models include 'R-CNN' names in torchvision [OK]

Common Mistakes:

• Confusing classification models with detection models
• Thinking Transformer is a torchvision detection model
• Mixing up model types

2. Which code snippet correctly loads a pretrained Faster R-CNN model from torchvision?

easy

A. model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights='DEFAULT')

B. model = torchvision.models.fasterrcnn_resnet50_fpn(pretrained=True)

C. model = torchvision.detection.fasterrcnn_resnet50_fpn(pretrained=True)

D. model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=False)

Solution

1. Step 1: Check the correct module and argument

   In recent torchvision versions, detection models are under torchvision.models.detection and use weights='DEFAULT' instead of pretrained=True.

2. Step 2: Identify the correct syntax

   model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights='DEFAULT') uses the correct module path and argument for loading pretrained weights.

3. Final Answer:

   model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights='DEFAULT') -> Option A

4. Quick Check:

   Use weights='DEFAULT' to load pretrained detection models [OK]

Hint: Use weights='DEFAULT' for pretrained detection models [OK]

Common Mistakes:

• Using pretrained=True instead of weights='DEFAULT'
• Wrong module path
• Missing 'models' in import path

3. What will be the output type of the following code snippet?

import torch
import torchvision
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights='DEFAULT')
model.eval()
inputs = [torch.rand(3, 300, 400)]
outputs = model(inputs)
print(type(outputs))

medium

A.

B.

C.

D.

Solution

1. Step 1: Understand model output format

   torchvision detection models return a list of dictionaries, one per input image.

2. Step 2: Check the type of outputs variable

   Since inputs is a list with one tensor, outputs will be a list containing one dictionary.

3. Final Answer:

   <class 'list'> -> Option D

4. Quick Check:

   Detection model output = list of dicts [OK]

Hint: Detection models output a list, one dict per input image [OK]

Common Mistakes:

• Expecting a single tensor output
• Confusing output with dict directly
• Not wrapping inputs in a list

4. Identify the error in this code snippet that tries to use a pretrained Mask R-CNN model:

import torchvision
model = torchvision.models.detection.maskrcnn_resnet50_fpn(pretrained=True)
model.eval()

medium

A. The function name should be fasterrcnn_resnet50_fpn

B. Mask R-CNN is not available in torchvision

C. The argument 'pretrained' is deprecated; use 'weights' instead

D. Missing import for torch

Solution

1. Step 1: Check function and argument usage

   Mask R-CNN is available as maskrcnn_resnet50_fpn in torchvision.models.detection.

2. Step 2: Identify deprecated argument

   Using pretrained=True is deprecated; the correct way is weights='DEFAULT'.

3. Final Answer:

   The argument 'pretrained' is deprecated; use 'weights' instead -> Option C

4. Quick Check:

   Use weights='DEFAULT' instead of pretrained=True [OK]

Hint: Replace pretrained=True with weights='DEFAULT' [OK]

Common Mistakes:

• Using deprecated pretrained argument
• Confusing model names
• Assuming Mask R-CNN is missing

5. You want to fine-tune a pretrained RetinaNet model from torchvision on your custom dataset with 5 classes. Which step is necessary before training?

hard

A. Replace the model's box predictor head to output 6 classes (5 + background)

B. Change the input image size to 224x224

C. Set pretrained=False to train from scratch

D. Freeze all layers except the backbone

Solution

1. Step 1: Understand class output adjustment

   Detection models include a background class, so for 5 classes, output head must predict 6 classes.

2. Step 2: Modify the box predictor head

   Replace the box predictor to match the number of classes (5 + 1 background) before training.

3. Final Answer:

   Replace the model's box predictor head to output 6 classes (5 + background) -> Option A

4. Quick Check:

   Adjust output head for custom classes [OK]

Hint: Add 1 for background class when changing output head [OK]

Common Mistakes:

• Not adjusting output head for new classes
• Changing input size unnecessarily
• Freezing all layers without fine-tuning head