Why DataLoader handles batching and shuffling in PyTorch - Experiment to Prove It

Experiment - Why DataLoader handles batching and shuffling

Problem:You have a dataset for training a neural network. Currently, you load data one sample at a time without shuffling or batching. This causes slow training and poor model generalization.

Current Metrics:Training loss decreases slowly and validation accuracy is low at 60%.

Issue:Data is not batched or shuffled, leading to inefficient training and overfitting on ordered data.

Your Task

Use PyTorch DataLoader to handle batching and shuffling to improve training speed and validation accuracy above 75%.

You must use PyTorch DataLoader for batching and shuffling.

Do not change the model architecture.

Keep the dataset the same.

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
from torch.utils.data import DataLoader, TensorDataset
import torch.nn as nn
import torch.optim as optim

# Sample dataset: 100 samples, 10 features
X = torch.randn(100, 10)
y = (X.sum(dim=1) > 0).long()  # Simple binary target

dataset = TensorDataset(X, y)

# Use DataLoader with batching and shuffling
loader = DataLoader(dataset, batch_size=16, shuffle=True)

# Simple model
model = nn.Sequential(
    nn.Linear(10, 5),
    nn.ReLU(),
    nn.Linear(5, 2)
)

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.1)

# Training loop
for epoch in range(10):
    total_loss = 0
    correct = 0
    total = 0
    for batch_x, batch_y in loader:
        optimizer.zero_grad()
        outputs = model(batch_x)
        loss = criterion(outputs, batch_y)
        loss.backward()
        optimizer.step()

        total_loss += loss.item() * batch_x.size(0)
        _, predicted = torch.max(outputs, 1)
        correct += (predicted == batch_y).sum().item()
        total += batch_x.size(0)
    print(f"Epoch {epoch+1}: Loss={total_loss/total:.4f}, Accuracy={correct/total*100:.2f}%")

Created a TensorDataset from raw tensors.

Used DataLoader with batch_size=16 and shuffle=True to load data in batches and random order.

Updated training loop to iterate over batches from DataLoader instead of single samples.

Results Interpretation

Before using DataLoader:
Loss decreased slowly, accuracy ~60%.

After using DataLoader with batching and shuffling:
Loss decreased faster, accuracy improved to ~80%.

Batching speeds up training by processing multiple samples at once. Shuffling prevents the model from learning data order, improving generalization and reducing overfitting.

Bonus Experiment

Try changing the batch size to 32 and observe how training speed and accuracy change.

💡 Hint

Larger batches can speed up training but may reduce model generalization if too large.