Epoch-based training in PyTorch

Introduction

Epoch-based training helps a model learn by showing it the entire dataset multiple times. This improves the model's ability to understand patterns.

When training a model on a dataset that fits in memory and you want to improve accuracy by multiple passes.

When you want to monitor how the model improves after each full pass through the data.

When using batch training and want to repeat the process several times to reduce errors.

When you want to control training time by setting a fixed number of passes over the data.

Syntax

PyTorch

for epoch in range(num_epochs):
    for inputs, labels in dataloader:
        # forward pass
        outputs = model(inputs)
        loss = loss_function(outputs, labels)
        
        # backward pass and optimization
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss.item():.4f}")

The outer loop runs over the number of epochs (full passes).

The inner loop runs over batches of data for each epoch.

Examples

This example shows a simple loop over 5 epochs with a print statement.

PyTorch

num_epochs = 5
for epoch in range(num_epochs):
    print(f"Starting epoch {epoch+1}")

This shows nested loops: epochs and batches inside each epoch.

PyTorch

for epoch in range(3):
    for batch in dataloader:
        # training steps here
        pass
    print(f"Completed epoch {epoch+1}")

Sample Model

This program trains a simple linear model to fit a line y = 3x + 2 with some noise. It runs for 5 epochs, printing the loss after each epoch.

PyTorch

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset

# Create a simple dataset
x = torch.linspace(-1, 1, 100).unsqueeze(1)
y = 3 * x + 2 + 0.1 * torch.randn(x.size())

# Dataset and DataLoader
dataset = TensorDataset(x, y)
dataloader = DataLoader(dataset, batch_size=10, shuffle=True)

# Simple linear model
model = nn.Linear(1, 1)

# Loss and optimizer
loss_function = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

num_epochs = 5
for epoch in range(num_epochs):
    for inputs, labels in dataloader:
        outputs = model(inputs)
        loss = loss_function(outputs, labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss.item():.4f}")

Important Notes

Loss usually decreases after each epoch if the model is learning well.

Shuffling data each epoch helps the model learn better by mixing data order.

Too many epochs can cause overfitting, where the model learns noise instead of patterns.

Summary

Epoch-based training means showing the whole dataset multiple times to the model.

Each epoch has many batches; the model updates after each batch.

Monitoring loss after each epoch helps track learning progress.