Why detection localizes objects in PyTorch - Experiment to Prove It

Experiment - Why detection localizes objects

Problem:You have a simple object detection model that predicts both the class and location (bounding box) of objects in images. Currently, the model predicts classes well but the bounding box predictions are poor, causing inaccurate localization.

Current Metrics:Training accuracy: 90%, Training bounding box IoU: 0.45; Validation accuracy: 88%, Validation bounding box IoU: 0.40

Issue:The model detects objects but does not localize them accurately. Bounding box Intersection over Union (IoU) is low, indicating poor localization.

Your Task

Improve the bounding box localization so that validation bounding box IoU increases to at least 0.60 without reducing classification accuracy below 85%.

Keep the model architecture simple (single-stage detector).

Do not increase training epochs beyond 30.

Do not change the dataset.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

PyTorch

import torch
import torch.nn as nn
import torch.optim as optim

class SimpleDetector(nn.Module):
    def __init__(self):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 16, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Conv2d(16, 32, 3, padding=1),
            nn.ReLU(),
            nn.MaxPool2d(2),
            nn.Flatten()
        )
        self.classifier = nn.Linear(32 * 64 * 64, 10)  # 10 classes
        self.bbox_regressor = nn.Linear(32 * 64 * 64, 4)  # bounding box coords

    def forward(self, x):
        x = self.features(x)
        class_logits = self.classifier(x)
        bbox_coords = self.bbox_regressor(x)
        return class_logits, bbox_coords

# Assume train_loader and val_loader are defined

model = SimpleDetector()
criterion_class = nn.CrossEntropyLoss()
criterion_bbox = nn.SmoothL1Loss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

num_epochs = 30
for epoch in range(num_epochs):
    model.train()
    for images, (labels, bboxes) in train_loader:
        optimizer.zero_grad()
        class_logits, bbox_preds = model(images)
        loss_class = criterion_class(class_logits, labels)
        loss_bbox = criterion_bbox(bbox_preds, bboxes)
        loss = loss_class + 5 * loss_bbox  # weight bbox loss
        loss.backward()
        optimizer.step()

    # Validation step omitted for brevity

# After training, evaluate classification accuracy and bounding box IoU on validation set
# (Evaluation code omitted for brevity)

Added a bounding box regression head to predict object locations.

Used Smooth L1 loss for bounding box regression to improve localization.

Combined classification loss and localization loss with a weighting factor.

Kept training epochs at 30 and used Adam optimizer for stable training.

Results Interpretation

Before: Validation accuracy 88%, bounding box IoU 0.40 (poor localization).

After: Validation accuracy 87%, bounding box IoU 0.61 (much better localization).

Adding a dedicated localization loss and balancing it with classification loss helps the model learn to predict object locations accurately without sacrificing classification performance.

Bonus Experiment

Try adding dropout layers in the feature extractor to reduce overfitting and see if localization improves further.

💡 Hint

Insert nn.Dropout(0.3) after convolutional layers and observe changes in validation bounding box IoU.

Practice

1. Why do object detection models predict bounding boxes along with class labels?

easy

A. To find the exact location of objects in the image

B. To increase the number of classes detected

C. To reduce the image size

D. To improve color accuracy

Solution

1. Step 1: Understand the role of bounding boxes

   Bounding boxes are used to mark the position and size of objects in an image.

2. Step 2: Connect bounding boxes to localization

   Predicting bounding boxes helps the model know where each object is located, not just what it is.

3. Final Answer:

   To find the exact location of objects in the image -> Option A

4. Quick Check:

   Bounding boxes = object location [OK]

Hint: Bounding boxes = object location in image [OK]

Common Mistakes:

• Thinking bounding boxes change image size
• Confusing bounding boxes with class labels
• Believing bounding boxes improve color

2. Which PyTorch output format correctly represents detection results with localization?

easy

A. (class_scores, bounding_boxes)

B. (image_tensor, class_labels)

C. (loss_value, optimizer_state)

D. (feature_maps, activation_maps)

Solution

1. Step 1: Identify detection output components

   Detection models output class scores and bounding boxes to classify and locate objects.

2. Step 2: Match output format to detection results

   The tuple (class_scores, bounding_boxes) correctly represents both classification and localization.

3. Final Answer:

   (class_scores, bounding_boxes) -> Option A

4. Quick Check:

   Detection output = (scores, boxes) [OK]

Hint: Detection outputs = class scores + bounding boxes [OK]

Common Mistakes:

• Confusing detection output with training loss
• Mixing input tensors with output results
• Using feature maps as final detection output

3. Given this PyTorch detection output code, what is the shape of boxes if batch size is 2 and max 3 detections per image?

class_scores, boxes = model(images)
print(boxes.shape)

medium

A. torch.Size([2, 4, 3])

B. torch.Size([3, 2, 4])

C. torch.Size([2, 3, 4])

D. torch.Size([4, 2, 3])

Solution

1. Step 1: Understand bounding box tensor shape

   Boxes are usually stored as (batch_size, max_detections, 4) where 4 = (x_min, y_min, x_max, y_max).

2. Step 2: Apply given values

   Batch size is 2, max detections 3, so shape is (2, 3, 4).

3. Final Answer:

   torch.Size([2, 3, 4]) -> Option C

4. Quick Check:

   Boxes shape = (batch, detections, 4) [OK]

Hint: Boxes shape = batch x detections x 4 coords [OK]

Common Mistakes:

• Swapping batch and detection dimensions
• Confusing coordinate count with detection count
• Assuming boxes shape matches class scores shape

4. You run a detection model but notice all predicted boxes are the same size and location. What is the likely cause?

medium

A. Batch size is too large

B. Bounding box regression layer is not trained properly

C. Input images are grayscale

D. Class scores are too low

Solution

1. Step 1: Identify the problem with bounding boxes

   If all boxes are identical, the model likely failed to learn how to adjust box coordinates.

2. Step 2: Connect to bounding box regression training

   The bounding box regression layer predicts box coordinates; if not trained well, outputs won't vary.

3. Final Answer:

   Bounding box regression layer is not trained properly -> Option B

4. Quick Check:

   Same boxes = bad box regression training [OK]

Hint: Same boxes mean box regression failed [OK]

Common Mistakes:

• Blaming class scores for box size issues
• Thinking grayscale images cause box errors
• Assuming batch size affects box variety

5. In a PyTorch object detection model, why is it important to predict both class scores and bounding boxes instead of only class scores?

hard

A. Because bounding boxes improve image resolution

B. Because bounding boxes reduce model size

C. Because class scores are not used in detection

D. Because class scores alone cannot tell where objects are located

Solution

1. Step 1: Understand the role of class scores

   Class scores tell what object is present but not its position in the image.

2. Step 2: Understand the role of bounding boxes

   Bounding boxes provide the exact location and size of the object, enabling localization.

3. Step 3: Combine both for full detection

   Detection requires knowing both what the object is and where it is, so both outputs are needed.

4. Final Answer:

   Because class scores alone cannot tell where objects are located -> Option D

5. Quick Check:

   Class scores = what, boxes = where [OK]

Hint: Class scores = what, boxes = where [OK]

Common Mistakes:

• Thinking bounding boxes affect image quality
• Believing class scores are unused in detection
• Assuming bounding boxes reduce model size