Data loading with torchvision in Computer Vision

Introduction

We use data loading with torchvision to easily get images ready for training AI models. It helps us organize and prepare pictures in a simple way.

When you want to train a model to recognize objects in photos.

When you need to load many images from folders for a project.

When you want to apply simple changes to images before training.

When you want to split images into batches for faster training.

Syntax

Computer Vision

from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# Define image transformations
transform = transforms.Compose([
    transforms.Resize((28, 28)),
    transforms.ToTensor()
])

# Load dataset
dataset = datasets.ImageFolder(root='path_to_images', transform=transform)

# Create data loader
loader = DataLoader(dataset, batch_size=32, shuffle=True)

datasets.ImageFolder loads images from folders where each folder name is a label.

transforms.Compose chains image changes like resizing and converting to numbers.

Examples

This example resizes images to 64x64 pixels and loads them in batches of 16.

Computer Vision

transform = transforms.Compose([
    transforms.Resize((64, 64)),
    transforms.ToTensor()
])
dataset = datasets.ImageFolder('data/train', transform=transform)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

This loads the CIFAR10 dataset directly with images converted to tensors.

Computer Vision

transform = transforms.ToTensor()
dataset = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(dataset, batch_size=64, shuffle=True)

Sample Model

This program loads the MNIST dataset, resizes images to 28x28, converts them to tensors, and loads them in batches of 32. It prints the shape of one batch and the first 5 labels.

Computer Vision

import torch
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

# Define transformations to resize and convert images to tensors
transform = transforms.Compose([
    transforms.Resize((28, 28)),
    transforms.ToTensor()
])

# Load MNIST dataset (handwritten digits)
dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)

# Create data loader with batch size 32
loader = DataLoader(dataset, batch_size=32, shuffle=True)

# Get one batch of images and labels
images, labels = next(iter(loader))

print(f'Batch image tensor shape: {images.shape}')
print(f'Batch labels tensor shape: {labels.shape}')
print(f'First 5 labels in batch: {labels[:5].tolist()}')

Important Notes

Always use transforms.ToTensor() to convert images to numbers the model can understand.

Shuffling data helps the model learn better by mixing images each time.

Batch size controls how many images the model sees at once; smaller batches use less memory.

Summary

Use torchvision's datasets and DataLoader to easily load and prepare image data.

Apply transforms to resize and convert images before training.

Load data in batches and shuffle to improve training efficiency and quality.

Practice

1. What is the main purpose of using torchvision.datasets in a computer vision project?

easy

A. To easily download and load popular image datasets

B. To create neural network layers

C. To visualize images in a dataset

D. To perform mathematical operations on tensors

Solution

1. Step 1: Understand the role of torchvision.datasets

   It provides ready-to-use popular image datasets like CIFAR10, MNIST, etc., for easy loading.

2. Step 2: Differentiate from other torchvision modules

   Other modules handle transforms or models, but datasets focus on loading data.

3. Final Answer:

   To easily download and load popular image datasets -> Option A

4. Quick Check:

   torchvision.datasets = load datasets [OK]

Hint: Datasets module is for loading data, not building models [OK]

Common Mistakes:

• Confusing datasets with model creation
• Thinking datasets handle image visualization
• Assuming datasets perform tensor math

2. Which of the following is the correct way to import the DataLoader class from torchvision?

easy

A. from torch.utils.data import DataLoader

B. from torchvision import DataLoader

C. import DataLoader from torchvision

D. from torchvision.datasets import DataLoader

Solution

1. Step 1: Recall the correct import path for DataLoader

   DataLoader is part of torch.utils.data, not torchvision directly.

2. Step 2: Check each option's syntax and source

   Only from torch.utils.data import DataLoader correctly imports DataLoader from torch.utils.data.

3. Final Answer:

   from torch.utils.data import DataLoader -> Option A

4. Quick Check:

   DataLoader import = torch.utils.data [OK]

Hint: DataLoader is in torch.utils.data, not torchvision [OK]

Common Mistakes:

• Importing DataLoader directly from torchvision
• Using incorrect import syntax
• Confusing datasets and DataLoader imports

3. What will be the output shape of images loaded from CIFAR10 dataset using torchvision if no transform is applied?

medium

A. [224, 224, 3]

B. [1, 28, 28]

C. [3, 32, 32]

D. [32, 32, 3]

Solution

1. Step 1: Recall CIFAR10 image dimensions

   CIFAR10 images are 32x32 pixels with 3 color channels (RGB).

2. Step 2: Understand PyTorch image tensor shape format

   PyTorch uses channel-first format: [channels, height, width], so shape is [3, 32, 32].

3. Final Answer:

   [3, 32, 32] -> Option C

4. Quick Check:

   CIFAR10 image shape = [3, 32, 32] [OK]

Hint: PyTorch images are channel-first: channels, height, width [OK]

Common Mistakes:

• Confusing channel order with height-width-channel
• Assuming grayscale images with 1 channel
• Mixing CIFAR10 size with MNIST or ImageNet

4. Identify the error in this code snippet for loading MNIST dataset with transforms:

from torchvision import datasets, transforms
transform = transforms.Compose([transforms.Resize(32), transforms.ToTensor()])
mnist_data = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(mnist_data, batch_size=64, shuffle=True)

medium

A. batch_size must be 1 or 32 only

B. Missing import of DataLoader from torch.utils.data

C. MNIST dataset does not support transforms

D. Transforms.Resize cannot resize images

Solution

1. Step 1: Check imports for DataLoader usage

   DataLoader is used but not imported, causing a NameError.

2. Step 2: Verify other parts of the code

   Transforms.Resize and MNIST support transforms; batch_size can be any positive integer.

3. Final Answer:

   Missing import of DataLoader from torch.utils.data -> Option B

4. Quick Check:

   DataLoader must be imported before use [OK]

Hint: Always import DataLoader before using it [OK]

Common Mistakes:

• Forgetting to import DataLoader
• Thinking MNIST doesn't support transforms
• Assuming Resize is invalid for MNIST

5. You want to load CIFAR10 images resized to 64x64 pixels, normalized with mean=[0.5,0.5,0.5] and std=[0.5,0.5,0.5], and shuffled in batches of 128. Which code snippet correctly achieves this?

hard

A. transform = transforms.Compose([transforms.Resize((64,64)), transforms.ToTensor()])
data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(data, batch_size=128, shuffle=True)

B. transform = transforms.Compose([transforms.ToTensor(), transforms.Resize(64), transforms.Normalize([0.5]*3, [0.5]*3)])
data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(data, batch_size=128, shuffle=False)

C. transform = transforms.Compose([transforms.Resize(64), transforms.Normalize([0.5]*3, [0.5]*3)])
data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(data, batch_size=64, shuffle=True)

D. transform = transforms.Compose([transforms.Resize((64,64)), transforms.ToTensor(), transforms.Normalize([0.5]*3, [0.5]*3)])
data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
loader = DataLoader(data, batch_size=128, shuffle=True)

Solution

1. Step 1: Check transform order and parameters

   Resize must be first with size (64,64), then ToTensor, then Normalize with correct mean and std.

2. Step 2: Verify DataLoader parameters

   Batch size is 128 and shuffle=True as required.

3. Step 3: Compare options

   transform = transforms.Compose([transforms.Resize((64,64)), transforms.ToTensor(), transforms.Normalize([0.5]*3, [0.5]*3)]) data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform) loader = DataLoader(data, batch_size=128, shuffle=True) matches all requirements exactly; others have wrong order, missing steps, or wrong batch/shuffle.

4. Final Answer:

   transform = transforms.Compose([transforms.Resize((64,64)), transforms.ToTensor(), transforms.Normalize([0.5]*3, [0.5]*3)]) data = datasets.CIFAR10(root='./data', train=True, download=True, transform=transform) loader = DataLoader(data, batch_size=128, shuffle=True) -> Option D

5. Quick Check:

   Resize->ToTensor->Normalize + batch=128 + shuffle=True [OK]

Hint: Resize first, then ToTensor, then Normalize; batch and shuffle as needed [OK]

Common Mistakes:

• Applying Normalize before ToTensor
• Using wrong Resize size or format
• Setting shuffle=False when shuffle=True needed
• Incorrect batch size