Kernel size, stride, padding in PyTorch

Introduction

Kernel size, stride, and padding control how a filter moves over an image in convolution. They help decide the output size and what parts of the image the model looks at.

When building a convolutional neural network to process images.

When you want to control how much the filter moves each step over the input.

When you want to keep the output image size the same as the input.

When you want to reduce the output size to focus on important features.

When you want to avoid losing edge information in images.

Syntax

PyTorch

torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0)

kernel_size is the size of the filter (e.g., 3 means 3x3).

stride is how many pixels the filter moves each step (default is 1).

padding adds pixels around the input edges (default is 0).

Examples

A 3x3 filter, stride 1, no padding.

PyTorch

conv = torch.nn.Conv2d(1, 10, kernel_size=3)

A 5x5 filter that moves 2 pixels each step, reducing output size.

PyTorch

conv = torch.nn.Conv2d(1, 10, kernel_size=5, stride=2)

A 3x3 filter with padding 1 to keep output size same as input.

PyTorch

conv = torch.nn.Conv2d(1, 10, kernel_size=3, padding=1)

Sample Model

This code shows how kernel size, stride, and padding affect output size and values. We use a simple 5x5 input and set all weights to 1 for clarity.

PyTorch

import torch
import torch.nn as nn

# Create a sample input: batch size 1, 1 channel, 5x5 image
input_tensor = torch.arange(25, dtype=torch.float32).reshape(1, 1, 5, 5)

# Define conv layers with different kernel_size, stride, padding
conv1 = nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=0)
conv2 = nn.Conv2d(1, 1, kernel_size=3, stride=2, padding=0)
conv3 = nn.Conv2d(1, 1, kernel_size=3, stride=1, padding=1)

# Initialize weights and bias to 1 for easy understanding
for conv in [conv1, conv2, conv3]:
    nn.init.constant_(conv.weight, 1.0)
    nn.init.constant_(conv.bias, 0.0)

# Apply convolutions
output1 = conv1(input_tensor)
output2 = conv2(input_tensor)
output3 = conv3(input_tensor)

# Print shapes and outputs
print(f"Output1 shape: {output1.shape}")
print(output1)
print(f"Output2 shape: {output2.shape}")
print(output2)
print(f"Output3 shape: {output3.shape}")
print(output3)

Important Notes

Padding helps keep the output size the same as input when using stride 1.

Stride greater than 1 reduces output size by skipping positions.

Kernel size controls the area the filter looks at each step.

Summary

Kernel size is the filter size that scans the input.

Stride controls how far the filter moves each step.

Padding adds pixels around input to control output size and edge info.