GAN training loop in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of discriminator loss calculation in GAN training loop

Consider the following snippet from a GAN training loop where the discriminator loss is calculated. What is the output of print(d_loss.item()) after one backward pass?

PyTorch

import torch
import torch.nn as nn

criterion = nn.BCELoss()
d_real = torch.tensor([0.9], requires_grad=True)
d_fake = torch.tensor([0.1], requires_grad=True)
real_labels = torch.ones_like(d_real)
fake_labels = torch.zeros_like(d_fake)

loss_real = criterion(d_real, real_labels)
loss_fake = criterion(d_fake, fake_labels)
d_loss = loss_real + loss_fake
d_loss.backward()
print(round(d_loss.item(), 3))

A 0.211

B 0.000

C 0.693

D 0.105

💡 Hint

Recall that BCELoss for a prediction close to the true label is low, and for wrong predictions is high.

The BCELoss for d_real=0.9 with label 1 is about 0.105, and for d_fake=0.1 with label 0 is about 0.105. Their sum is approximately 0.211.

❓ Model Choice

intermediate

Choosing the correct optimizer update step in GAN training

In a GAN training loop, after computing the discriminator loss and calling d_loss.backward(), which of the following is the correct next step to update the discriminator's weights?

A loss.backward()

B optimizer_d.zero_grad()

C optimizer_d.step()

D optimizer_g.step()

💡 Hint

Think about which optimizer corresponds to the discriminator.

After backward(), calling optimizer_d.step() updates the discriminator weights. zero_grad() clears gradients and should be called before backward().

❓ Hyperparameter

advanced

Effect of learning rate on GAN training stability

Which of the following learning rate choices is most likely to cause unstable GAN training with mode collapse?

A Learning rate = 0.0005

B Learning rate = 0.0002

C Learning rate = 0.00005

D Learning rate = 0.1

💡 Hint

High learning rates can cause large weight updates and instability.

A learning rate of 0.1 is too high for GANs and often causes unstable training and mode collapse.

🔧 Debug

advanced

Identifying the bug in GAN generator training step

In the generator training step below, which line causes the generator to not learn properly?

PyTorch

optimizer_g.zero_grad()
fake_data = generator(noise)
d_fake = discriminator(fake_data)
g_loss = criterion(d_fake, real_labels)
g_loss.backward()
optimizer_d.step()  # <-- suspicious line

A g_loss.backward()

B optimizer_d.step()

C optimizer_g.zero_grad()

D criterion(d_fake, real_labels)

💡 Hint

Check which optimizer should update the generator weights.

Calling optimizer_d.step() updates discriminator weights instead of generator weights, so generator does not learn.

🧠 Conceptual

expert

Why alternate training of discriminator and generator is important in GANs

Why do GAN training loops alternate between updating the discriminator and the generator instead of updating both simultaneously?

A To prevent one network from overpowering the other, maintaining a balance for stable training

B Because simultaneous updates cause syntax errors in PyTorch

C To reduce the total training time by skipping some updates

D Because the generator requires more frequent updates than the discriminator

💡 Hint

Think about the adversarial nature of GANs and training stability.

Alternating updates keep discriminator and generator balanced, preventing one from dominating and causing unstable training.