Train/val/test split in PyTorch - Model Pipeline Trace

Model Pipeline - Train/val/test split

This pipeline splits the dataset into three parts: training, validation, and testing. The training set teaches the model, the validation set helps tune it, and the test set checks how well it learned.

Data Flow - 2 Stages

1Original Dataset

1000 rows x 10 columns→Start with full dataset→1000 rows x 10 columns

[[5.1, 3.5, ..., 0], [4.9, 3.0, ..., 1], ...]

↓

2Train/Val/Test Split

1000 rows x 10 columns→Split dataset into 70% train, 15% val, 15% test→Train: 700 rows x 10 columns, Val: 150 rows x 10 columns, Test: 150 rows x 10 columns

Train sample: [5.1, 3.5, ..., 0], Val sample: [6.7, 3.1, ..., 1], Test sample: [5.9, 3.0, ..., 0]

Training Trace - Epoch by Epoch

Loss
1.0 |****
0.8 |****
0.6 |****
0.4 |****
0.2 |****
0.0 +----
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.60	Model starts learning with moderate loss and accuracy.
2	0.65	0.72	Loss decreases and accuracy improves as model learns.
3	0.50	0.80	Model shows good progress with lower loss and higher accuracy.
4	0.40	0.85	Training continues to improve with steady loss decrease.
5	0.35	0.88	Model converges with low loss and high accuracy.

Prediction Trace - 3 Layers

Layer 1: Input Sample

Layer 2: Model Forward Pass

Layer 3: Prediction

Model Quiz - 3 Questions

Why do we split data into train, validation, and test sets?

A To train, tune, and fairly evaluate the model

B To increase dataset size

C To reduce model complexity

D To speed up training only

Key Insight

Splitting data into train, validation, and test sets helps the model learn well, tune hyperparameters, and fairly measure performance on unseen data.

Practice

1. What is the main purpose of splitting data into train, validation, and test sets in PyTorch?

easy

A. To train the model, tune hyperparameters, and evaluate performance separately

B. To increase the size of the dataset artificially

C. To reduce the number of features in the dataset

D. To convert data into tensors automatically

Solution

1. Step 1: Understand the role of each data split

   The training set is used to teach the model, the validation set helps tune settings, and the test set checks how well the model works on new data.

2. Step 2: Match purpose to options

   Only To train the model, tune hyperparameters, and evaluate performance separately correctly describes these roles. Other options describe unrelated tasks.

3. Final Answer:

   To train the model, tune hyperparameters, and evaluate performance separately -> Option A

4. Quick Check:

   Train/val/test split = separate training, tuning, testing [OK]

Hint: Remember: train teaches, val tunes, test checks [OK]

Common Mistakes:

• Thinking splitting increases dataset size
• Confusing splitting with feature selection
• Assuming splitting converts data types

2. Which of the following is the correct way to split a PyTorch dataset into train and test sets using random_split?

easy

A. train_set, test_set = random_split(dataset, [80%, 20%])

B. train_set, test_set = random_split(dataset, [80, 20])

C. train_set, test_set = random_split(dataset, [0.8, 0.2])

D. train_set, test_set = random_split(dataset, [80])

Solution

1. Step 1: Check the expected input for random_split

   random_split expects the dataset and a list of lengths (integers) for each split, not fractions or percentages.

2. Step 2: Evaluate each option

   train_set, test_set = random_split(dataset, [80, 20]) uses integer lengths [80, 20], which is correct. train_set, test_set = random_split(dataset, [0.8, 0.2]) uses floats, train_set, test_set = random_split(dataset, [80%, 20%]) uses invalid syntax with %, and train_set, test_set = random_split(dataset, [80]) provides only one length.

3. Final Answer:

   train_set, test_set = random_split(dataset, [80, 20]) -> Option B

4. Quick Check:

   random_split needs integer lengths [OK]

Hint: Use integer lengths, not fractions, in random_split [OK]

Common Mistakes:

• Passing fractions or percentages instead of integers
• Providing only one length instead of two
• Using invalid syntax like % in list

3. Given the code below, what will be the length of val_set?

from torch.utils.data import random_split

full_dataset = list(range(100))
train_size = 70
val_size = 20
test_size = 10
train_set, val_set, test_set = random_split(full_dataset, [train_size, val_size, test_size])
print(len(val_set))

medium

A. 10

B. 70

C. 20

D. 100

Solution

1. Step 1: Understand random_split lengths

   The list [70, 20, 10] tells random_split to split the dataset into parts of those exact lengths.

2. Step 2: Identify val_set length

   val_set corresponds to the second split with length 20, so len(val_set) will be 20.

3. Final Answer:

   20 -> Option C

4. Quick Check:

   val_size = 20 means val_set length = 20 [OK]

Hint: Lengths in random_split match output sizes exactly [OK]

Common Mistakes:

• Confusing sizes with percentages
• Assuming val_set length is sum of others
• Forgetting order of splits in output

4. You wrote this code to split your dataset but get a ValueError: Sum of input lengths does not equal the length of the input dataset!. What is the error?

from torch.utils.data import random_split

dataset = list(range(50))
train_set, test_set = random_split(dataset, [30, 25])

medium

A. The sum of lengths 30 + 25 is greater than dataset length 50

B. random_split requires fractions, not lengths

C. dataset must be a PyTorch tensor, not a list

D. random_split cannot split into two parts

Solution

1. Step 1: Check dataset length and split lengths

   Dataset length is 50. The sum of split lengths is 30 + 25 = 55, which is more than 50.

2. Step 2: Understand random_split requirements

   random_split requires the sum of split lengths to exactly match dataset length.

3. Final Answer:

   The sum of lengths 30 + 25 is greater than dataset length 50 -> Option A

4. Quick Check:

   Sum of split lengths must equal dataset length [OK]

Hint: Sum of split lengths must equal dataset size exactly [OK]

Common Mistakes:

• Passing split lengths that don't sum to dataset size
• Thinking random_split accepts fractions
• Assuming dataset must be tensor, not list

5. You want to split a PyTorch dataset of 1000 samples into 70% train, 15% validation, and 15% test sets. Which code correctly performs this split using random_split?

hard

A. train_set, val_set, test_set = random_split(dataset, [0.7, 0.15, 0.15])

B. train_set, val_set, test_set = random_split(dataset, [700, 150, 100])

C. train_set, val_set, test_set = random_split(dataset, [700, 150])

D. train_set, val_set, test_set = random_split(dataset, [700, 150, 150])

Solution

1. Step 1: Calculate integer split sizes

   70% of 1000 is 700, 15% is 150, so splits are [700, 150, 150].

2. Step 2: Check options for correctness

   train_set, val_set, test_set = random_split(dataset, [700, 150, 150]) uses correct integer lengths summing to 1000. train_set, val_set, test_set = random_split(dataset, [0.7, 0.15, 0.15]) uses floats (invalid), train_set, val_set, test_set = random_split(dataset, [700, 150]) misses one split, train_set, val_set, test_set = random_split(dataset, [700, 150, 100]) sums to 950 (incorrect).

3. Final Answer:

   train_set, val_set, test_set = random_split(dataset, [700, 150, 150]) -> Option D

4. Quick Check:

   Use integer lengths summing to dataset size [OK]

Hint: Convert percentages to integers summing to dataset size [OK]

Common Mistakes:

• Passing fractions instead of integers
• Not summing split sizes to dataset length
• Missing one of the splits