Why custom loops offer full control in TensorFlow - Test Your Understanding

Practice - 5 Tasks

1fill in blank

easy

Complete the code to start a custom training loop by iterating over the dataset.

TensorFlow

for [1] in dataset:
    # training step here

Drag options to blanks, or click blank then click option'

Abatch

Bmodel

Closs

Doptimizer

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'model' or 'loss' as the loop variable instead of data batches.

We use batch to represent each group of data from the dataset in the loop.

2fill in blank

medium

Complete the code to compute the loss inside the custom training loop.

TensorFlow

with tf.GradientTape() as tape:
    predictions = model([1])
    loss = loss_fn(labels, predictions)

Drag options to blanks, or click blank then click option'

Abatch

Blabels

Coptimizer

Dloss

Attempts:

3 left

💡 Hint

Common Mistakes

Passing labels or optimizer instead of input data to the model.

The model takes the input data batch to make predictions.

3fill in blank

hard

Fix the error in the code to apply gradients to the model variables.

TensorFlow

gradients = tape.gradient(loss, [1].trainable_variables)
optimizer.apply_gradients(zip(gradients, model.trainable_variables))

Drag options to blanks, or click blank then click option'

Apredictions

Bmodel

Coptimizer

Dloss

Attempts:

3 left

💡 Hint

Common Mistakes

Using loss or optimizer instead of model for trainable variables.

Gradients are computed with respect to the model's trainable variables.

4fill in blank

hard

Fill both blanks to update the training accuracy metric inside the loop.

TensorFlow

train_accuracy.[1](labels, [2])

Drag options to blanks, or click blank then click option'

Aupdate_state

Breset_states

Cpredictions

Dresult

Attempts:

3 left

💡 Hint

Common Mistakes

Using reset_states instead of update_state, or passing wrong arguments.

We update the accuracy metric state with the true labels and the model's predictions.

5fill in blank

hard

Fill all three blanks to complete the custom training loop with loss calculation, gradient application, and metric update.

TensorFlow

with tf.GradientTape() as tape:
    predictions = model([1])
    loss = loss_fn(labels, predictions)
gradients = tape.gradient(loss, model.[2])
optimizer.apply_gradients(zip(gradients, model.trainable_variables))
train_accuracy.[3](labels, predictions)

Drag options to blanks, or click blank then click option'

Abatch

Btrainable_variables

Cupdate_state

Dloss

Attempts:

3 left

💡 Hint

Common Mistakes

Mixing up variables for gradient calculation or metric update method.

The batch is input to the model, gradients are computed on trainable variables, and the accuracy metric is updated with labels and predictions.

Practice

1. Why do custom training loops in TensorFlow give you full control over the training process?

easy

A. Because they require no coding and run faster by default

B. Because TensorFlow automatically handles all training details for you

C. Because custom loops use less memory than built-in training methods

D. Because you manually define each step like forward pass, loss calculation, and backpropagation

Solution

1. Step 1: Understand what custom loops do

   Custom loops let you write code for each training step, like calculating predictions and losses.

2. Step 2: Compare with automatic training

   Built-in methods hide these steps, but custom loops expose them for full control.

3. Final Answer:

   Because you manually define each step like forward pass, loss calculation, and backpropagation -> Option D

4. Quick Check:

   Manual step control = Full control [OK]

Hint: Custom loops mean you write every training step yourself [OK]

Common Mistakes:

• Thinking TensorFlow handles all details automatically
• Assuming custom loops run faster by default
• Believing custom loops require no coding

2. Which of the following is the correct syntax to start a custom training loop in TensorFlow?

easy

A. train_loop(num_epochs):

B. while epoch < num_epochs {}

C. for epoch in range(num_epochs):

D. loop(epoch in num_epochs)

Solution

1. Step 1: Identify Python loop syntax

   In Python, a for loop over a range is written as for variable in range(...):.

2. Step 2: Match with TensorFlow custom loop

   Custom loops use standard Python loops like for epoch in range(num_epochs): to iterate epochs.

3. Final Answer:

   for epoch in range(num_epochs): -> Option C

4. Quick Check:

   Python for loop syntax = for epoch in range(num_epochs): [OK]

Hint: Custom loops use normal Python for loops for epochs [OK]

Common Mistakes:

• Using invalid loop syntax like curly braces
• Confusing function calls with loops
• Using incorrect keywords like 'loop' or 'while' without colon

3. Consider this snippet of a TensorFlow custom training loop:

for epoch in range(2):
    for x, y in dataset:
        with tf.GradientTape() as tape:
            predictions = model(x)
            loss = loss_fn(y, predictions)
        gradients = tape.gradient(loss, model.trainable_variables)
        optimizer.apply_gradients(zip(gradients, model.trainable_variables))
print(loss.numpy())

What will be printed after training?

medium

A. The loss value from the last batch of the second epoch

B. The total loss summed over all batches

C. The initial loss before training

D. The average loss over all batches

Solution

1. Step 1: Understand the loop structure

   The loop runs 2 epochs, updating loss each batch. The printed loss is from the last batch processed.

2. Step 2: Identify what loss.numpy() prints

   It prints the loss tensor's value from the last batch, not a sum or average.

3. Final Answer:

   The loss value from the last batch of the second epoch -> Option A

4. Quick Check:

   Printed loss = last batch loss [OK]

Hint: Printed loss after loop is last batch's loss [OK]

Common Mistakes:

• Thinking loss is summed or averaged automatically
• Assuming loss is from first batch
• Confusing loss tensor with scalar summary

4. You wrote this custom training loop but get an error:

for epoch in range(3):
    for x, y in dataset:
        with tf.GradientTape() as tape:
            predictions = model(x)
            loss = loss_fn(y, predictions)
        gradients = tape.gradient(loss, model.variables)
        optimizer.apply_gradients(zip(gradients, model.trainable_variables))

What is the likely cause of the error?

medium

A. Missing optimizer initialization

B. Using model.variables instead of model.trainable_variables in gradient calculation

C. Not calling loss.numpy() before gradient calculation

D. Forgetting to reset gradients after each batch

Solution

1. Step 1: Check gradient calculation variables

   Gradients must be computed with respect to trainable variables, not all variables.

2. Step 2: Identify the error source

   Using model.variables includes non-trainable variables, causing mismatch in optimizer step.

3. Final Answer:

   Using model.variables instead of model.trainable_variables in gradient calculation -> Option B

4. Quick Check:

   Gradient variables must be trainable variables [OK]

Hint: Use model.trainable_variables for gradients, not model.variables [OK]

Common Mistakes:

• Using all variables instead of trainable ones
• Forgetting to initialize optimizer
• Trying to convert loss to numpy before gradient step

5. You want to implement a custom training loop that stops early if validation loss does not improve for 3 epochs. Which approach gives you full control to do this?

hard

A. Write a for loop over epochs, compute training and validation loss each epoch, track best validation loss, and break loop if no improvement for 3 epochs

B. Use model.fit() with early stopping callback

C. Train with model.evaluate() and stop manually

D. Use a while loop without tracking validation loss

Solution

1. Step 1: Understand early stopping logic

   You must track validation loss each epoch and count epochs without improvement.

2. Step 2: Implement control flow in custom loop

   Using a for loop with manual checks and break statements gives full control to stop early.

3. Final Answer:

   Write a for loop over epochs, compute training and validation loss each epoch, track best validation loss, and break loop if no improvement for 3 epochs -> Option A

4. Quick Check:

   Manual loop + validation tracking = full control [OK]

Hint: Use manual loop with validation checks and break for early stopping [OK]

Common Mistakes:

• Relying only on model.fit() without custom logic
• Not tracking validation loss properly
• Using infinite loops without break conditions