Epoch-based training in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of a simple epoch training loop

What will be the printed output after running this PyTorch training loop for 2 epochs?

Assume the model, optimizer, and loss function are correctly defined and the dataloader has 3 batches with dummy data.

Code:

PyTorch

for epoch in range(2):
    total_loss = 0
    for batch in range(3):
        loss = batch + 1  # dummy loss values: 1, 2, 3
        total_loss += loss
    print(f"Epoch {epoch+1}, Loss: {total_loss}")

A

Epoch 1, Loss: 6
Epoch 2, Loss: 6

B

Epoch 1, Loss: 3
Epoch 2, Loss: 3

C

Epoch 1, Loss: 1
Epoch 2, Loss: 2

D

Epoch 1, Loss: 6
Epoch 2, Loss: 9

💡 Hint

Sum the dummy loss values for each batch inside the epoch.

The dummy loss values per batch are 1, 2, and 3. Their sum is 6. This sum is printed for each epoch, so both epochs print 6.

❓ Model Choice

intermediate

Choosing the correct place to reset loss in epoch training

In an epoch-based training loop, where should you reset the total loss variable to zero to correctly track loss per epoch?

A Before the start of each batch loop

B Before the start of each epoch loop

C After the end of all epochs

D Inside the optimizer step

💡 Hint

Think about when you want to start fresh loss tracking for each epoch.

Resetting total loss before each epoch ensures loss is accumulated fresh for that epoch only.

❓ Hyperparameter

advanced

Effect of increasing number of epochs on model training

What is the most likely effect of increasing the number of epochs during training of a neural network?

A Model may overfit the training data if epochs are too high

B Model will always perform better on unseen data

C Training time decreases as epochs increase

D Loss will increase on training data as epochs increase

💡 Hint

Think about what happens if the model learns too much from training data.

Too many epochs can cause the model to memorize training data, reducing its ability to generalize to new data.

❓ Metrics

advanced

Calculating average loss per epoch

Given a training loop that sums batch losses in total_loss and processes 5 batches per epoch, how do you calculate the average loss per epoch?

A Multiply total_loss by number of batches (5)

B Subtract number of batches (5) from total_loss

C Add number of batches (5) to total_loss

D Divide total_loss by number of batches (5)

💡 Hint

Average means total divided by count.

Average loss per epoch is total loss divided by number of batches processed in that epoch.

🔧 Debug

expert

Identifying the bug in epoch training loop

What error will this PyTorch epoch training loop raise?

Code:
for epoch in range(3): total_loss = 0 for batch in dataloader: optimizer.zero_grad() outputs = model(batch[0]) loss = criterion(outputs, batch[1]) loss.backward() optimizer.step() total_loss += loss print(f"Epoch {epoch+1} Loss: {total_loss}")

A RuntimeError: loss.backward() called multiple times

B NameError: name 'optimizer' is not defined

C TypeError: unsupported operand type(s) for +=: 'int' and 'Tensor'

D No error, code runs correctly

💡 Hint

Check the type of loss and total_loss before adding.

loss is a tensor, total_loss is an int. Adding them directly causes a TypeError.

Practice

1. What does one epoch represent in PyTorch training?

easy

A. One full pass through the entire training dataset

B. One batch of data processed by the model

C. One update of the model weights

D. One evaluation of the model on the test set

Solution

1. Step 1: Understand the definition of an epoch

   An epoch means the model sees every example in the training dataset once.

2. Step 2: Differentiate epoch from batch and update

   A batch is a subset of data; multiple batches make an epoch. Weight updates happen after each batch, not the whole dataset.

3. Final Answer:

   One full pass through the entire training dataset -> Option A

4. Quick Check:

   Epoch = full dataset pass [OK]

Hint: Epoch means full dataset seen once by model [OK]

Common Mistakes:

• Confusing batch with epoch
• Thinking epoch is one weight update
• Mixing test evaluation with training epoch

2. Which PyTorch code snippet correctly sets up a loop for 5 epochs?

easy

A. for epoch in range(1, 5):

B. for epoch in range(5, 10):

C. for epoch in range(0, 6):

D. for epoch in range(5):

Solution

1. Step 1: Recall Python range behavior

   range(5) generates 0,1,2,3,4 which is 5 iterations.

2. Step 2: Check other options for iteration count

   range(1,5) gives 4 iterations; range(0,6) gives 6; range(5,10) gives 5 but starts at 5, not zero.

3. Final Answer:

   for epoch in range(5): -> Option D

4. Quick Check:

   range(5) = 5 loops [OK]

Hint: range(5) loops exactly 5 times from 0 to 4 [OK]

Common Mistakes:

• Using range(1,5) which loops 4 times
• Using range(0,6) which loops 6 times
• Starting loop at 5 instead of 0

3. Given this PyTorch training loop snippet, what will be printed after 2 epochs?

for epoch in range(2):
    total_loss = 0
    for batch in dataloader:
        output = model(batch)
        loss = loss_fn(output, batch.labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        total_loss += loss.item()
    print(f"Epoch {epoch+1} loss: {total_loss}")

medium

A. Loss values for all batches printed

B. Loss values for epoch 1 and epoch 2 printed

C. Only loss for epoch 2 printed

D. No output printed due to error

Solution

1. Step 1: Analyze the outer loop and print statement

   The outer loop runs 2 times (epochs 0 and 1). After each epoch, it prints total_loss.

2. Step 2: Understand total_loss accumulation and printing

   total_loss sums batch losses per epoch, so print shows loss after each epoch.

3. Final Answer:

   Loss values for epoch 1 and epoch 2 printed -> Option B

4. Quick Check:

   Print inside epoch loop prints each epoch loss [OK]

Hint: Print inside epoch loop shows loss per epoch [OK]

Common Mistakes:

• Thinking only last epoch loss prints
• Confusing batch loss with epoch loss
• Assuming print causes error

4. Identify the error in this epoch-based training code snippet:

for epoch in range(3):
    for batch in dataloader:
        output = model(batch)
        loss = loss_fn(output, batch.labels)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
    print(f"Epoch {epoch+1} done")

medium

A. loss.backward() should be called after optimizer.step()

B. optimizer.step() should be called after optimizer.zero_grad()

C. optimizer.zero_grad() should be called before loss.backward()

D. No error, code is correct

Solution

1. Step 1: Recall correct order of optimizer calls

   Zero gradients before backward pass to avoid accumulating gradients.

2. Step 2: Check code order

   Code calls loss.backward() before optimizer.zero_grad(), which is wrong.

3. Final Answer:

   optimizer.zero_grad() should be called before loss.backward() -> Option C

4. Quick Check:

   Zero grads before backward [OK]

Hint: Always zero gradients before backward pass [OK]

Common Mistakes:

• Zeroing gradients after backward
• Swapping optimizer.step() and zero_grad() order
• Thinking no zero_grad needed

5. You want to train a model for 10 epochs but stop early if validation loss does not improve for 3 epochs. Which approach fits best with epoch-based training?

hard

A. After each epoch, check validation loss and stop if no improvement for 3 epochs

B. Stop training after 3 batches with no improvement

C. Train all 10 epochs regardless of validation loss

D. Check validation loss only after all 10 epochs finish

Solution

1. Step 1: Understand early stopping concept

   Early stopping monitors validation loss after each epoch to decide if training should continue.

2. Step 2: Match early stopping with epoch-based training

   Checking validation loss after each epoch and stopping if no improvement for 3 epochs fits epoch-based training.

3. Final Answer:

   After each epoch, check validation loss and stop if no improvement for 3 epochs -> Option A

4. Quick Check:

   Early stopping checks validation loss per epoch [OK]

Hint: Early stop checks validation loss after each epoch [OK]

Common Mistakes:

• Checking after batches instead of epochs
• Ignoring validation loss during training
• Waiting until all epochs finish to check