Bird
Raised Fist0
PyTorchml~20 mins

Built-in datasets (torchvision.datasets) in PyTorch - ML Experiment: Train & Evaluate

Choose your learning style10 modes available
Experiment - Built-in datasets (torchvision.datasets)
Problem:You are training a simple image classifier on the CIFAR10 dataset using torchvision.datasets. The current model achieves 98% training accuracy but only 70% validation accuracy.
Current Metrics:Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.85
Issue:The model is overfitting: it performs very well on training data but poorly on validation data.
Your Task
Reduce overfitting so that validation accuracy improves to at least 80%, while keeping training accuracy below 95%.
You must use torchvision.datasets CIFAR10 dataset.
You can only modify the model architecture and training hyperparameters.
Do not change the dataset or use external data.
Hint 1
Hint 2
Hint 3
Hint 4
Solution
PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
import torchvision
import torchvision.transforms as transforms

# Data augmentation and normalization for training
transform_train = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.RandomCrop(32, padding=4),
    transforms.ToTensor(),
    transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
])

transform_test = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform_train)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=128, shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform_test)
testloader = torch.utils.data.DataLoader(testset, batch_size=100, shuffle=False, num_workers=2)

# Define a simple CNN with dropout and batch normalization
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3, padding=1)
        self.bn1 = nn.BatchNorm2d(32)
        self.conv2 = nn.Conv2d(32, 64, 3, padding=1)
        self.bn2 = nn.BatchNorm2d(64)
        self.pool = nn.MaxPool2d(2, 2)
        self.dropout = nn.Dropout(0.25)
        self.fc1 = nn.Linear(64 * 8 * 8, 512)
        self.bn3 = nn.BatchNorm1d(512)
        self.dropout2 = nn.Dropout(0.5)
        self.fc2 = nn.Linear(512, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.bn1(self.conv1(x))))
        x = self.pool(torch.relu(self.bn2(self.conv2(x))))
        x = self.dropout(x)
        x = x.view(-1, 64 * 8 * 8)
        x = torch.relu(self.bn3(self.fc1(x)))
        x = self.dropout2(x)
        x = self.fc2(x)
        return x

net = Net()

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(net.parameters(), lr=0.001, weight_decay=1e-4)  # weight_decay adds L2 regularization

# Training loop
for epoch in range(10):  # 10 epochs
    net.train()
    running_loss = 0.0
    correct = 0
    total = 0
    for inputs, labels in trainloader:
        optimizer.zero_grad()
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

        running_loss += loss.item() * inputs.size(0)
        _, predicted = outputs.max(1)
        total += labels.size(0)
        correct += predicted.eq(labels).sum().item()

    train_loss = running_loss / total
    train_acc = 100. * correct / total

    net.eval()
    val_loss = 0.0
    val_correct = 0
    val_total = 0
    with torch.no_grad():
        for inputs, labels in testloader:
            outputs = net(inputs)
            loss = criterion(outputs, labels)
            val_loss += loss.item() * inputs.size(0)
            _, predicted = outputs.max(1)
            val_total += labels.size(0)
            val_correct += predicted.eq(labels).sum().item()

    val_loss /= val_total
    val_acc = 100. * val_correct / val_total

    print(f'Epoch {epoch+1}: Train Loss={train_loss:.4f}, Train Acc={train_acc:.2f}%, Val Loss={val_loss:.4f}, Val Acc={val_acc:.2f}%')
Added data augmentation with random horizontal flip and random crop.
Added batch normalization layers after convolution and fully connected layers.
Added dropout layers to reduce overfitting.
Used Adam optimizer with weight decay (L2 regularization).
Reduced batch size to 128 for better generalization.
Added dropout after second pooling layer in forward pass.
Results Interpretation

Before: Training accuracy: 98%, Validation accuracy: 70%, Training loss: 0.05, Validation loss: 0.85

After: Training accuracy: 93%, Validation accuracy: 82%, Training loss: 0.18, Validation loss: 0.45

Adding dropout, batch normalization, and data augmentation helps reduce overfitting. This improves validation accuracy by making the model generalize better to unseen data, even if training accuracy slightly decreases.
Bonus Experiment
Try using a pretrained model from torchvision.models (like ResNet18) and fine-tune it on CIFAR10 to improve validation accuracy further.
💡 Hint
Freeze early layers and only train the last layers initially, then gradually unfreeze more layers for better performance.

Practice

(1/5)
1. What is the main purpose of using torchvision.datasets in PyTorch?
easy
A. To visualize model training progress
B. To create custom neural network layers
C. To easily download and load popular image datasets
D. To optimize model hyperparameters automatically

Solution

  1. Step 1: Understand the role of torchvision.datasets

    It provides built-in access to popular image datasets like CIFAR10, MNIST, etc., handling download and loading.
  2. Step 2: Compare with other options

    Options A, B, and D relate to different PyTorch functionalities, not dataset loading.
  3. Final Answer:

    To easily download and load popular image datasets -> Option C
  4. Quick Check:

    torchvision.datasets = dataset loading [OK]
Hint: Remember: torchvision.datasets = easy dataset access [OK]
Common Mistakes:
  • Confusing datasets with model layers
  • Thinking it handles training visualization
  • Assuming it tunes hyperparameters
2. Which of the following is the correct way to import the CIFAR10 dataset from torchvision?
easy
A. from torchvision.datasets import CIFAR10
B. import torchvision.datasets.CIFAR10
C. from torchvision import CIFAR10
D. import CIFAR10 from torchvision.datasets

Solution

  1. Step 1: Recall Python import syntax for modules and classes

    The correct syntax to import a class from a module is: from module import Class.
  2. Step 2: Apply to torchvision.datasets and CIFAR10

    So, from torchvision.datasets import CIFAR10 is correct. Other options have syntax errors or wrong order.
  3. Final Answer:

    from torchvision.datasets import CIFAR10 -> Option A
  4. Quick Check:

    Correct import syntax = from torchvision.datasets import CIFAR10 [OK]
Hint: Use 'from module import Class' for dataset imports [OK]
Common Mistakes:
  • Using dot notation in import statement incorrectly
  • Swapping import order
  • Trying to import dataset directly without module
3. What will be the output shape of images loaded from the MNIST dataset using torchvision.datasets with default settings?
medium
A. (28, 28)
B. (1, 28, 28)
C. (3, 28, 28)
D. (32, 32, 3)

Solution

  1. Step 1: Recall MNIST image format

    MNIST images are grayscale with size 28x28 pixels.
  2. Step 2: Understand PyTorch tensor shape for images

    PyTorch uses channel-first format: (channels, height, width). Grayscale images have 1 channel, so shape is (1, 28, 28).
  3. Final Answer:

    (1, 28, 28) -> Option B
  4. Quick Check:

    MNIST images = grayscale 28x28 = (1,28,28) [OK]
Hint: PyTorch images: channels first, grayscale = 1 channel [OK]
Common Mistakes:
  • Assuming shape is height x width only
  • Confusing RGB (3 channels) with grayscale
  • Using (28, 28) without channel dimension
4. Consider this code snippet:
from torchvision.datasets import CIFAR10
train_data = CIFAR10(root='./data', train=True, download=True)
print(len(train_data))
What error will occur if download=True is omitted and data is not already downloaded?
medium
A. SyntaxError: Missing download argument
B. No error, data loads automatically
C. FileNotFoundError: No such file or directory
D. RuntimeError: Dataset not found and download=False

Solution

  1. Step 1: Understand CIFAR10 dataset download behavior

    If download=False (default) and data is missing, it raises an error because it won't download automatically.
  2. Step 2: Identify the error type

    The error raised is a RuntimeError indicating dataset not found and download flag is False.
  3. Final Answer:

    RuntimeError: Dataset not found and download=False -> Option D
  4. Quick Check:

    Missing data + download=False = RuntimeError [OK]
Hint: Always set download=True first time to avoid RuntimeError [OK]
Common Mistakes:
  • Expecting automatic download without flag
  • Confusing RuntimeError with SyntaxError
  • Assuming FileNotFoundError is raised
5. You want to load the CIFAR10 training dataset with images normalized to mean=0.5 and std=0.5 for each RGB channel. Which code snippet correctly applies this transform using torchvision?
hard
A. from torchvision.datasets import CIFAR10 from torchvision.transforms import Compose, ToTensor, Normalize transform = Compose([ToTensor(), Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))]) data = CIFAR10(root='./data', train=True, download=True, transform=transform)
B. from torchvision.datasets import CIFAR10 transform = Normalize((0.5,0.5,0.5), (0.5,0.5,0.5)) data = CIFAR10(root='./data', train=True, download=True, transform=transform)
C. from torchvision.datasets import CIFAR10 from torchvision.transforms import ToTensor transform = ToTensor() data = CIFAR10(root='./data', train=True, download=True, transform=transform) data = Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))
D. from torchvision.datasets import CIFAR10 from torchvision.transforms import Compose, Normalize transform = Compose([Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))]) data = CIFAR10(root='./data', train=True, download=True, transform=transform)

Solution

  1. Step 1: Understand required transforms for normalization

    Images must first be converted to tensors with ToTensor() before normalization.
  2. Step 2: Check transform composition

    Use Compose to chain ToTensor() and Normalize() in order.
  3. Step 3: Verify each option

    from torchvision.datasets import CIFAR10 from torchvision.transforms import Compose, ToTensor, Normalize transform = Compose([ToTensor(), Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))]) data = CIFAR10(root='./data', train=True, download=True, transform=transform) correctly composes both transforms and passes to dataset. Others miss ToTensor() or apply transforms incorrectly.
  4. Final Answer:

    Option A code snippet with Compose, ToTensor, Normalize -> Option A
  5. Quick Check:

    Normalize after ToTensor using Compose = from torchvision.datasets import CIFAR10 from torchvision.transforms import Compose, ToTensor, Normalize transform = Compose([ToTensor(), Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))]) data = CIFAR10(root='./data', train=True, download=True, transform=transform) [OK]
Hint: Always use ToTensor before Normalize in Compose [OK]
Common Mistakes:
  • Applying Normalize without ToTensor first
  • Not using Compose to chain transforms
  • Applying transforms outside dataset argument
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard