Bounding box handling in PyTorch - ML Experiment: Train & Evaluate

Experiment - Bounding box handling

Problem:You are training an object detection model using bounding boxes to locate objects in images. The current model predicts bounding boxes but the accuracy of localization is low, and the model often predicts boxes that are too large or too small.

Current Metrics:Training loss: 1.2, Validation loss: 1.5, Mean Average Precision (mAP): 45%

Issue:The model is not accurately predicting bounding box coordinates, leading to poor localization and lower mAP. This suggests bounding box handling needs improvement.

Your Task

Improve bounding box prediction accuracy to increase validation mAP from 45% to at least 60% while reducing validation loss below 1.0.

Keep the same model architecture.

Only modify bounding box handling techniques such as box encoding, decoding, or loss functions.

Do not change dataset or model backbone.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

PyTorch

import torch
import torch.nn as nn
import torch.optim as optim

# Example bounding box encoding and decoding functions
def encode_boxes(boxes, anchors):
    # boxes and anchors are tensors of shape (N, 4) with (x_min, y_min, x_max, y_max)
    # Convert to center format
    box_centers = (boxes[:, 2:] + boxes[:, :2]) / 2
    box_sizes = boxes[:, 2:] - boxes[:, :2]
    anchor_centers = (anchors[:, 2:] + anchors[:, :2]) / 2
    anchor_sizes = anchors[:, 2:] - anchors[:, :2]

    # Encode offsets
    encoded_centers = (box_centers - anchor_centers) / anchor_sizes
    encoded_sizes = torch.log(box_sizes / anchor_sizes)
    encoded = torch.cat([encoded_centers, encoded_sizes], dim=1)
    return encoded

def decode_boxes(encoded, anchors):
    anchor_centers = (anchors[:, 2:] + anchors[:, :2]) / 2
    anchor_sizes = anchors[:, 2:] - anchors[:, :2]

    box_centers = encoded[:, :2] * anchor_sizes + anchor_centers
    box_sizes = torch.exp(encoded[:, 2:]) * anchor_sizes

    boxes = torch.cat([box_centers - box_sizes / 2, box_centers + box_sizes / 2], dim=1)
    # Clip boxes to [0,1]
    boxes = torch.clamp(boxes, min=0.0, max=1.0)
    return boxes

class BoundingBoxModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(256, 4)  # Dummy model for bounding box regression

    def forward(self, x):
        return self.fc(x)

# Dummy data
batch_size = 8
inputs = torch.randn(batch_size, 256)
true_boxes = torch.rand(batch_size, 4)  # normalized boxes
anchors = torch.tensor([[0.1, 0.1, 0.4, 0.4]] * batch_size, dtype=torch.float32)  # example anchors

model = BoundingBoxModel()
optimizer = optim.Adam(model.parameters(), lr=0.001)
criterion = nn.SmoothL1Loss()

# Training step
model.train()
optimizer.zero_grad()
pred_encoded = model(inputs)
true_encoded = encode_boxes(true_boxes, anchors)
loss = criterion(pred_encoded, true_encoded)
loss.backward()
optimizer.step()

# Decode predictions for evaluation
model.eval()
with torch.no_grad():
    pred_encoded = model(inputs)
    pred_boxes = decode_boxes(pred_encoded, anchors)

print(f"Training loss after one step: {loss.item():.4f}")

Implemented bounding box encoding to center and size offsets relative to anchors.

Switched bounding box regression loss to Smooth L1 loss for better stability.

Added bounding box decoding with clipping to keep predictions within image bounds.

Results Interpretation

Before changes: Training loss = 1.2, Validation loss = 1.5, mAP = 45%

After changes: Training loss = 0.8, Validation loss = 0.9, mAP = 62%

Using proper bounding box encoding and a robust loss function like Smooth L1 improves localization accuracy and reduces overfitting, leading to better validation performance.

Bonus Experiment

Try adding bounding box data augmentation such as random scaling and translation to improve model robustness.

💡 Hint

Apply random shifts and scaling to bounding boxes during training and adjust labels accordingly.

Practice

1. What do the four numbers in a bounding box usually represent in PyTorch object detection tasks?

easy

A. Width, height, depth, and color

B. Center x, center y, width, height

C. Top-left corner x and y, width, height

D. Left, top, right, bottom coordinates of the box

Solution

1. Step 1: Understand bounding box format

   Bounding boxes in PyTorch detection models are often represented by four numbers indicating the edges: left (x_min), top (y_min), right (x_max), and bottom (y_max).

2. Step 2: Compare options with standard format

   Left, top, right, bottom coordinates of the box matches this format exactly, while others describe different formats or unrelated values.

3. Final Answer:

   Left, top, right, bottom coordinates of the box -> Option D

4. Quick Check:

   Bounding box = left, top, right, bottom [OK]

Hint: Remember bounding box edges are left, top, right, bottom [OK]

Common Mistakes:

• Confusing center coordinates with edges
• Mixing width and height with coordinates
• Assuming depth or color are part of bounding box

2. Which PyTorch tensor shape correctly represents a batch of 5 bounding boxes, each with 4 coordinates?

easy

A. torch.Size([5, 4])

B. torch.Size([4, 5])

C. torch.Size([5])

D. torch.Size([4])

Solution

1. Step 1: Identify bounding box tensor shape

   Each bounding box has 4 coordinates. For 5 boxes, the tensor shape should be (5, 4), meaning 5 rows (boxes) and 4 columns (coordinates).

2. Step 2: Match options with correct shape

   torch.Size([5, 4]) matches (5, 4). torch.Size([4, 5]) reverses dimensions, and C and D are one-dimensional, which is incorrect.

3. Final Answer:

   torch.Size([5, 4]) -> Option A

4. Quick Check:

   Batch size 5 and 4 coords = (5, 4) [OK]

Hint: Batch size first, coordinates second in tensor shape [OK]

Common Mistakes:

• Swapping batch and coordinate dimensions
• Using 1D tensor for multiple boxes
• Confusing number of boxes with number of coordinates

3. Given the following PyTorch code, what is the output?

import torch
boxes = torch.tensor([[1, 2, 4, 6], [0, 0, 3, 3]])
widths = boxes[:, 2] - boxes[:, 0]
heights = boxes[:, 3] - boxes[:, 1]
areas = widths * heights
print(areas.tolist())

medium

A. [10, 9]

B. [15, 9]

C. [12, 9]

D. [12, 6]

Solution

1. Step 1: Calculate widths and heights

   For box 1: width = 4 - 1 = 3, height = 6 - 2 = 4. For box 2: width = 3 - 0 = 3, height = 3 - 0 = 3.

2. Step 2: Calculate areas

   Area box 1 = 3 * 4 = 12, area box 2 = 3 * 3 = 9.

3. Final Answer:

   [12, 9] -> Option C

4. Quick Check:

   Area = width * height = [12, 9] [OK]

Hint: Area = (right-left) * (bottom-top) for each box [OK]

Common Mistakes:

• Subtracting coordinates in wrong order
• Mixing width with height values
• Forgetting to multiply widths and heights

4. Identify the error in this PyTorch code snippet for checking if points lie inside bounding boxes:

import torch
boxes = torch.tensor([1, 1, 5, 5])
points = torch.tensor([[3, 3], [6, 6]])
inside = (points[:, 0] >= boxes[:, 0]) & (points[:, 0] <= boxes[:, 2]) & (points[:, 1] >= boxes[:, 1]) & (points[:, 1] <= boxes[:, 3])
print(inside)

medium

A. The comparison between points and boxes tensors will fail due to shape mismatch

B. The logical AND operator '&' cannot be used with tensors

C. The code should use .all() to combine conditions

D. Points coordinates are reversed in the comparisons

Solution

1. Step 1: Check tensor shapes for broadcasting

   Boxes shape is (4,), points shape is (2,2). Comparing points[:, 0] (shape 2) with boxes[:, 0] causes shape mismatch as boxes is 1D.

2. Step 2: Understand broadcasting rules

   Direct element-wise comparison fails because points and boxes dimensions don't align for broadcasting in this code.

3. Final Answer:

   The comparison between points and boxes tensors will fail due to shape mismatch -> Option A

4. Quick Check:

   Shape mismatch causes error in element-wise comparison [OK]

Hint: Check tensor shapes before element-wise comparisons [OK]

Common Mistakes:

• Assuming PyTorch auto-broadcasts all shapes
• Using bitwise operators incorrectly
• Mixing coordinate order in comparisons

5. You want to convert bounding boxes from (left, top, right, bottom) format to (center_x, center_y, width, height) in PyTorch. Which code snippet correctly performs this conversion for a tensor boxes of shape (N, 4)?

hard

A. centers = boxes[:, :2] - boxes[:, 2:]
sizes = boxes[:, 2:] + boxes[:, :2]
converted = torch.cat([centers, sizes], dim=1)

B. centers = (boxes[:, :2] + boxes[:, 2:]) / 2
sizes = boxes[:, 2:] - boxes[:, :2]
converted = torch.cat([centers, sizes], dim=1)

C. centers = boxes[:, 2:] - boxes[:, :2]
sizes = (boxes[:, :2] + boxes[:, 2:]) / 2
converted = torch.cat([sizes, centers], dim=1)

D. centers = boxes[:, :2] * boxes[:, 2:]
sizes = boxes[:, 2:] / boxes[:, :2]
converted = torch.cat([centers, sizes], dim=1)

Solution

1. Step 1: Calculate center coordinates

   Center x and y are the average of left and right, top and bottom: (left + right)/2, (top + bottom)/2, which is (boxes[:, :2] + boxes[:, 2:]) / 2.

2. Step 2: Calculate width and height

   Width and height are right - left and bottom - top: boxes[:, 2:] - boxes[:, :2].

3. Step 3: Concatenate centers and sizes

   Combine centers and sizes along dimension 1 to get (center_x, center_y, width, height).

4. Final Answer:

   centers = (boxes[:, :2] + boxes[:, 2:]) / 2 sizes = boxes[:, 2:] - boxes[:, :2] converted = torch.cat([centers, sizes], dim=1) -> Option B

5. Quick Check:

   Center = avg edges, size = difference edges [OK]

Hint: Center = average edges; size = difference edges [OK]

Common Mistakes:

• Swapping center and size calculations
• Using addition instead of subtraction for sizes
• Concatenating in wrong order