Train/val/test split in PyTorch

Introduction

Splitting data into train, validation, and test sets helps us build models that learn well and check how good they are on new data.

When you want to teach a model and check its learning progress.

When you want to tune model settings without cheating by looking at test data.

When you want to see how well your model works on data it has never seen.

When you have a dataset and want to avoid overfitting.

When you want to compare different models fairly.

Syntax

PyTorch

from sklearn.model_selection import train_test_split

# Split data into train and temp (val+test)
train_data, temp_data = train_test_split(data, test_size=0.4, random_state=42)

# Split temp_data into validation and test
val_data, test_data = train_test_split(temp_data, test_size=0.5, random_state=42)

We first split data into training and a temporary set (which will be split again).

Then we split the temporary set equally into validation and test sets.

Examples

Splits data into 70% train, 15% val, and 15% test.

PyTorch

train_data, temp_data = train_test_split(data, test_size=0.3)
val_data, test_data = train_test_split(temp_data, test_size=0.5)

Splits data into 80% train and 20% validation only.

PyTorch

train_data, val_data = train_test_split(data, test_size=0.2)
# No separate test set here

Sample Model

This code creates a simple dataset and splits it into train, validation, and test sets using PyTorch's random_split. It then prints the number of samples in each set.

PyTorch

import torch
from torch.utils.data import DataLoader, random_split, TensorDataset

# Create dummy dataset of 100 samples with 5 features
X = torch.randn(100, 5)
y = torch.randint(0, 2, (100,))

# Combine features and labels into a TensorDataset
dataset = TensorDataset(X, y)

# Define split sizes: 70 train, 15 val, 15 test
train_size = 70
val_size = 15
test_size = 15

# Use random_split to split dataset
train_dataset, val_dataset, test_dataset = random_split(dataset, [train_size, val_size, test_size], generator=torch.Generator().manual_seed(42))

# Create DataLoaders
train_loader = DataLoader(train_dataset, batch_size=10)
val_loader = DataLoader(val_dataset, batch_size=10)
test_loader = DataLoader(test_dataset, batch_size=10)

# Print sizes to confirm
print(f"Train samples: {len(train_dataset)}")
print(f"Validation samples: {len(val_dataset)}")
print(f"Test samples: {len(test_dataset)}")

Important Notes

Always set a random seed to get the same split every time you run your code.

Validation data helps tune your model without touching test data.

Test data should only be used once at the end to check final model performance.

Summary

Split your data into three parts: train, validation, and test.

Train data teaches the model, validation data tunes it, and test data checks it.

Use tools like sklearn's train_test_split or PyTorch's random_split for easy splitting.

Practice

1. What is the main purpose of splitting data into train, validation, and test sets in PyTorch?

easy

A. To train the model, tune hyperparameters, and evaluate performance separately

B. To increase the size of the dataset artificially

C. To reduce the number of features in the dataset

D. To convert data into tensors automatically

Solution

1. Step 1: Understand the role of each data split

   The training set is used to teach the model, the validation set helps tune settings, and the test set checks how well the model works on new data.

2. Step 2: Match purpose to options

   Only To train the model, tune hyperparameters, and evaluate performance separately correctly describes these roles. Other options describe unrelated tasks.

3. Final Answer:

   To train the model, tune hyperparameters, and evaluate performance separately -> Option A

4. Quick Check:

   Train/val/test split = separate training, tuning, testing [OK]

Hint: Remember: train teaches, val tunes, test checks [OK]

Common Mistakes:

• Thinking splitting increases dataset size
• Confusing splitting with feature selection
• Assuming splitting converts data types

2. Which of the following is the correct way to split a PyTorch dataset into train and test sets using random_split?

easy

A. train_set, test_set = random_split(dataset, [80%, 20%])

B. train_set, test_set = random_split(dataset, [80, 20])

C. train_set, test_set = random_split(dataset, [0.8, 0.2])

D. train_set, test_set = random_split(dataset, [80])

Solution

1. Step 1: Check the expected input for random_split

   random_split expects the dataset and a list of lengths (integers) for each split, not fractions or percentages.

2. Step 2: Evaluate each option

   train_set, test_set = random_split(dataset, [80, 20]) uses integer lengths [80, 20], which is correct. train_set, test_set = random_split(dataset, [0.8, 0.2]) uses floats, train_set, test_set = random_split(dataset, [80%, 20%]) uses invalid syntax with %, and train_set, test_set = random_split(dataset, [80]) provides only one length.

3. Final Answer:

   train_set, test_set = random_split(dataset, [80, 20]) -> Option B

4. Quick Check:

   random_split needs integer lengths [OK]

Hint: Use integer lengths, not fractions, in random_split [OK]

Common Mistakes:

• Passing fractions or percentages instead of integers
• Providing only one length instead of two
• Using invalid syntax like % in list

3. Given the code below, what will be the length of val_set?

from torch.utils.data import random_split

full_dataset = list(range(100))
train_size = 70
val_size = 20
test_size = 10
train_set, val_set, test_set = random_split(full_dataset, [train_size, val_size, test_size])
print(len(val_set))

medium

A. 10

B. 70

C. 20

D. 100

Solution

1. Step 1: Understand random_split lengths

   The list [70, 20, 10] tells random_split to split the dataset into parts of those exact lengths.

2. Step 2: Identify val_set length

   val_set corresponds to the second split with length 20, so len(val_set) will be 20.

3. Final Answer:

   20 -> Option C

4. Quick Check:

   val_size = 20 means val_set length = 20 [OK]

Hint: Lengths in random_split match output sizes exactly [OK]

Common Mistakes:

• Confusing sizes with percentages
• Assuming val_set length is sum of others
• Forgetting order of splits in output

4. You wrote this code to split your dataset but get a ValueError: Sum of input lengths does not equal the length of the input dataset!. What is the error?

from torch.utils.data import random_split

dataset = list(range(50))
train_set, test_set = random_split(dataset, [30, 25])

medium

A. The sum of lengths 30 + 25 is greater than dataset length 50

B. random_split requires fractions, not lengths

C. dataset must be a PyTorch tensor, not a list

D. random_split cannot split into two parts

Solution

1. Step 1: Check dataset length and split lengths

   Dataset length is 50. The sum of split lengths is 30 + 25 = 55, which is more than 50.

2. Step 2: Understand random_split requirements

   random_split requires the sum of split lengths to exactly match dataset length.

3. Final Answer:

   The sum of lengths 30 + 25 is greater than dataset length 50 -> Option A

4. Quick Check:

   Sum of split lengths must equal dataset length [OK]

Hint: Sum of split lengths must equal dataset size exactly [OK]

Common Mistakes:

• Passing split lengths that don't sum to dataset size
• Thinking random_split accepts fractions
• Assuming dataset must be tensor, not list

5. You want to split a PyTorch dataset of 1000 samples into 70% train, 15% validation, and 15% test sets. Which code correctly performs this split using random_split?

hard

A. train_set, val_set, test_set = random_split(dataset, [0.7, 0.15, 0.15])

B. train_set, val_set, test_set = random_split(dataset, [700, 150, 100])

C. train_set, val_set, test_set = random_split(dataset, [700, 150])

D. train_set, val_set, test_set = random_split(dataset, [700, 150, 150])

Solution

1. Step 1: Calculate integer split sizes

   70% of 1000 is 700, 15% is 150, so splits are [700, 150, 150].

2. Step 2: Check options for correctness

   train_set, val_set, test_set = random_split(dataset, [700, 150, 150]) uses correct integer lengths summing to 1000. train_set, val_set, test_set = random_split(dataset, [0.7, 0.15, 0.15]) uses floats (invalid), train_set, val_set, test_set = random_split(dataset, [700, 150]) misses one split, train_set, val_set, test_set = random_split(dataset, [700, 150, 100]) sums to 950 (incorrect).

3. Final Answer:

   train_set, val_set, test_set = random_split(dataset, [700, 150, 150]) -> Option D

4. Quick Check:

   Use integer lengths summing to dataset size [OK]

Hint: Convert percentages to integers summing to dataset size [OK]

Common Mistakes:

• Passing fractions instead of integers
• Not summing split sizes to dataset length
• Missing one of the splits