Training loop structure in PyTorch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to start the training loop for 5 epochs.

PyTorch

for epoch in range([1]):
    print(f"Epoch {epoch + 1}")

Drag options to blanks, or click blank then click option'

A10

B0

C5

D1

Attempts:

3 left

💡 Hint

Common Mistakes

Using range(10) which runs too many epochs.

Using range(0) which runs zero times.

The training loop runs for 5 epochs, so the range should be 5.

2fill in blank

medium

Complete the code to zero the gradients before backpropagation.

PyTorch

optimizer.[1]()

Drag options to blanks, or click blank then click option'

Aupdate

Bstep

Cbackward

Dzero_grad

Attempts:

3 left

💡 Hint

Common Mistakes

Calling optimizer.step() before zeroing gradients.

Using optimizer.backward() which does not exist.

Before backpropagation, gradients must be reset using optimizer.zero_grad().

3fill in blank

hard

Fix the error in the code to perform backpropagation on the loss.

PyTorch

loss.[1]()

Drag options to blanks, or click blank then click option'

Abackward

Bupdate

Czero_grad

Dstep

Attempts:

3 left

💡 Hint

Common Mistakes

Calling loss.step() which does not exist.

Calling loss.zero_grad() which is incorrect.

To compute gradients, call loss.backward() after computing the loss.

4fill in blank

hard

Fill both blanks to update model parameters and print the loss value.

PyTorch

optimizer.[1]()
print(f"Loss: {loss.[2]()}")

Drag options to blanks, or click blank then click option'

Astep

Bitem

Cbackward

Dzero_grad

Attempts:

3 left

💡 Hint

Common Mistakes

Calling loss.backward() instead of loss.item() in print.

Calling optimizer.zero_grad() instead of optimizer.step().

optimizer.step() updates parameters. loss.item() gets the scalar loss value for printing.

5fill in blank

hard

Fill all three blanks to complete the training loop: zero gradients, compute loss backward, and update parameters.

PyTorch

for epoch in range(3):
    optimizer.[1]()
    output = model(data)
    loss = loss_fn(output, target)
    loss.[2]()
    optimizer.[3]()

Drag options to blanks, or click blank then click option'

Azero_grad

Bbackward

Cstep

Deval

Attempts:

3 left

💡 Hint

Common Mistakes

Calling optimizer.eval() which is for evaluation mode.

Skipping zero_grad causing gradient accumulation.

In each epoch, zero gradients, compute gradients with backward(), then update parameters with step().

Practice

1. Which of the following best describes the main purpose of a training loop in PyTorch?

easy

A. To save the model to a file

B. To repeatedly update the model weights to improve performance

C. To visualize the model architecture

D. To load data from disk into memory

Solution

1. Step 1: Understand the training loop role

   The training loop runs many times to adjust model weights based on data.

2. Step 2: Identify the main goal

   Its goal is to improve model performance by updating weights repeatedly.

3. Final Answer:

   To repeatedly update the model weights to improve performance -> Option B

4. Quick Check:

   Training loop = update weights repeatedly [OK]

Hint: Training loop = repeat updates to improve model [OK]

Common Mistakes:

• Confusing data loading with training loop
• Thinking training loop saves model
• Assuming training loop only visualizes model

2. Which of the following is the correct way to zero gradients in a PyTorch training loop?

easy

A. optimizer.zero_grad()

B. model.zero_gradients()

C. loss.zero_grad()

D. torch.zero_grad()

Solution

1. Step 1: Recall PyTorch gradient clearing method

   Gradients are cleared by calling zero_grad() on the optimizer.

2. Step 2: Check each option

   Only optimizer.zero_grad() is the correct PyTorch syntax.

3. Final Answer:

   optimizer.zero_grad() -> Option A

4. Quick Check:

   Clear gradients with optimizer.zero_grad() [OK]

Hint: Clear gradients with optimizer.zero_grad() [OK]

Common Mistakes:

• Calling zero_grad on model or loss
• Using non-existent torch.zero_grad()
• Forgetting to zero gradients before backward pass

3. What will be the output of the following PyTorch training loop snippet?

for epoch in range(2):
    for batch in range(3):
        print(f"Epoch {epoch}, Batch {batch}")

medium

A. Epoch 0, Batch 0
Epoch 0, Batch 1
Epoch 1, Batch 0
Epoch 1, Batch 1

B. Epoch 0, Batch 0
Epoch 1, Batch 0
Epoch 2, Batch 0

C. Epoch 0, Batch 0
Epoch 0, Batch 1
Epoch 0, Batch 2

D. Epoch 0, Batch 0
Epoch 0, Batch 1
Epoch 0, Batch 2
Epoch 1, Batch 0
Epoch 1, Batch 1
Epoch 1, Batch 2

Solution

1. Step 1: Understand loop ranges

   Outer loop runs for epochs 0 and 1 (2 times). Inner loop runs for batches 0,1,2 (3 times).

2. Step 2: Trace output lines

   For each epoch, batches 0 to 2 print, so total 6 lines with epoch and batch numbers.

3. Final Answer:

   Epoch 0, Batch 0 Epoch 0, Batch 1 Epoch 0, Batch 2 Epoch 1, Batch 0 Epoch 1, Batch 1 Epoch 1, Batch 2 -> Option D

4. Quick Check:

   2 epochs x 3 batches = 6 lines printed [OK]

Hint: Multiply epochs by batches for total prints [OK]

Common Mistakes:

• Confusing range end values
• Mixing epoch and batch indices
• Counting fewer batches than loop runs

4. Identify the error in this PyTorch training loop snippet:

for data, target in dataloader:
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output, target)
    loss.backward
    optimizer.step()

medium

A. Missing parentheses after loss.backward

B. optimizer.zero_grad() should be after optimizer.step()

C. criterion should be called before model(data)

D. optimizer.step() should be called before loss.backward

Solution

1. Step 1: Check method calls

   loss.backward is missing parentheses, so it is not called as a function.

2. Step 2: Verify correct order

   optimizer.zero_grad() is correctly before backward and step; criterion call order is correct; optimizer.step() after backward is correct.

3. Final Answer:

   Missing parentheses after loss.backward -> Option A

4. Quick Check:

   Call loss.backward() with parentheses [OK]

Hint: Always add () to call backward() [OK]

Common Mistakes:

• Forgetting parentheses on backward()
• Changing correct optimizer.zero_grad() position
• Misordering loss and model calls

5. You want to train a model for 5 epochs with batches of size 10. Which of the following training loop structures correctly implements this in PyTorch?

hard

A. for epoch in range(5):
optimizer.zero_grad()
for batch_idx, (data, target) in enumerate(dataloader):
output = model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()

B. for batch_idx, (data, target) in enumerate(dataloader):
for epoch in range(5):
optimizer.zero_grad()
output = model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()

C. for epoch in range(5):
for batch_idx, (data, target) in enumerate(dataloader):
optimizer.zero_grad()
output = model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()

D. for epoch in range(5):
for batch_idx, (data, target) in enumerate(dataloader):
output = model(data)
loss = criterion(output, target)
optimizer.zero_grad()
loss.backward()
optimizer.step()

Solution

1. Step 1: Check loop order

   The outer loop should be epochs, inner loop batches. for epoch in range(5): for batch_idx, (data, target) in enumerate(dataloader): optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() follows this correctly.

2. Step 2: Verify zero_grad and step calls

   optimizer.zero_grad() must be before backward(), and optimizer.step() after. for epoch in range(5): for batch_idx, (data, target) in enumerate(dataloader): optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() has correct order.

3. Step 3: Check other options

   for batch_idx, (data, target) in enumerate(dataloader): for epoch in range(5): optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() reverses loops (wrong). for epoch in range(5): optimizer.zero_grad() for batch_idx, (data, target) in enumerate(dataloader): output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() zeros gradients outside batch loop (wrong). for epoch in range(5): for batch_idx, (data, target) in enumerate(dataloader): output = model(data) loss = criterion(output, target) optimizer.zero_grad() loss.backward() optimizer.step() zeros gradients after forward pass (wrong order).

4. Final Answer:

   for epoch in range(5): for batch_idx, (data, target) in enumerate(dataloader): optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() -> Option C

5. Quick Check:

   Epochs outside, zero_grad before backward, step after [OK]

Hint: Epoch loop outside, zero_grad before backward [OK]

Common Mistakes:

• Swapping epoch and batch loops
• Calling zero_grad outside batch loop
• Calling zero_grad after forward pass