Training and validation loss tracking in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is training loss in machine learning?

Training loss measures how well the model fits the training data. It shows the error the model makes on the data it learns from.

beginner

Why do we track validation loss during training?

Validation loss helps check if the model is learning to generalize well to new data, not just memorizing the training data.

intermediate

What does it mean if training loss keeps decreasing but validation loss starts increasing?

This usually means the model is overfitting: it learns the training data too well but performs worse on new data.

beginner

How can you track training and validation loss in PyTorch?

You calculate loss on training batches during training and on validation batches after each epoch, then save these values to plot or analyze.

beginner

What is a common way to visualize training and validation loss?

Plotting loss values over epochs on a line graph helps see trends and detect overfitting or underfitting.

What does a decreasing training loss indicate?

A The model is not learning

B The model is overfitting

C The model is learning from the training data

D The validation data is incorrect

Decreasing training loss means the model is improving its predictions on the training data.

If validation loss increases while training loss decreases, what is likely happening?

A Underfitting

B Overfitting

C Perfect fit

D Data leakage

Increasing validation loss with decreasing training loss usually means overfitting.

When should validation loss be calculated during training?

A Only at the end of training

B Before training starts

C After each training batch

D After each epoch

Validation loss is typically calculated after each epoch to evaluate model generalization.

Which PyTorch function is commonly used to compute loss?

A torch.nn.CrossEntropyLoss()

B torch.save()

C torch.tensor()

D torch.optim.Adam()

torch.nn.CrossEntropyLoss() is a common loss function for classification tasks.

What is the best way to detect overfitting using loss values?

A Training loss decreases, validation loss increases

B Training and validation loss both increase

C Both losses stay constant

D Validation loss decreases faster than training loss

Overfitting is indicated when training loss decreases but validation loss increases.

Explain why tracking both training and validation loss is important during model training.

Think about how the model performs on data it has seen versus new data.

Describe how you would implement loss tracking in a PyTorch training loop.

Consider when and how often you compute loss during training.