Kernel size, stride, padding in PyTorch - ML Experiment: Train & Evaluate

Experiment - Kernel size, stride, padding

Problem:You have a convolutional neural network layer that processes images. The current layer uses a kernel size of 3, stride of 1, and no padding. The output feature map is smaller than expected, causing loss of important edge information.

Current Metrics:Input image size: 28x28, Output feature map size: 26x26

Issue:The output feature map is smaller than the input, which may cause loss of edge information. This happens because no padding is used and stride is 1 with kernel size 3.

Your Task

Adjust the kernel size, stride, and padding to keep the output feature map size the same as the input (28x28) while maintaining stride 1.

Stride must remain 1

Kernel size can be 3 or 5

Padding must be adjusted accordingly

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
import torch.nn as nn

# Define a convolutional layer with kernel size 3, stride 1, padding 1
conv_layer = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=3, stride=1, padding=1)

# Create a dummy input tensor with batch size 1, 1 channel, 28x28 image
input_tensor = torch.randn(1, 1, 28, 28)

# Pass input through conv layer
output_tensor = conv_layer(input_tensor)

# Print output shape
print(f"Output shape with kernel=3, stride=1, padding=1: {output_tensor.shape}")

# Now try kernel size 5 with padding 2
conv_layer_5 = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=5, stride=1, padding=2)
output_tensor_5 = conv_layer_5(input_tensor)
print(f"Output shape with kernel=5, stride=1, padding=2: {output_tensor_5.shape}")

Added padding to keep output size same as input

Used padding=1 for kernel size 3

Used padding=2 for kernel size 5

Kept stride fixed at 1

Results Interpretation

Before: Output size was 26x26 with kernel=3, stride=1, padding=0.
After: Output size is 28x28 with kernel=3, stride=1, padding=1 or kernel=5, stride=1, padding=2.

Padding controls the size of the output feature map. By adding appropriate padding, you can keep the output size the same as the input, preserving edge information. Kernel size and padding must be balanced to achieve this.

Bonus Experiment

Try changing the stride to 2 with kernel size 3 and padding 1. Observe how the output size changes.

💡 Hint

Use the output size formula and see how stride affects the output dimensions. Expect the output size to reduce roughly by half.