Simple GAN implementation in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

Output of discriminator prediction on real images

Given a simple GAN discriminator model and a batch of real images, what is the shape of the output predictions?

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers

def make_discriminator_model():
    model = tf.keras.Sequential([
        layers.Flatten(input_shape=(28, 28, 1)),
        layers.Dense(512, activation='relu'),
        layers.Dense(1, activation='sigmoid')
    ])
    return model

# Create discriminator
D = make_discriminator_model()

# Batch of 16 real images (28x28 grayscale)
real_images = tf.random.normal([16, 28, 28, 1])

# Get predictions
predictions = D(real_images)

print(predictions.shape)

A (1, 16)

B (16, 1)

C (16, 28, 28, 1)

D (16,)

💡 Hint

The discriminator outputs one probability per input image.

The discriminator outputs a single probability (shape 1) for each image in the batch. Since the batch size is 16, the output shape is (16, 1).

❓ Model Choice

intermediate

Choosing the correct generator output shape

Which generator model architecture correctly outputs a 28x28 grayscale image from a 100-dimensional noise vector?

A Sequential model with Dense(7*7*128), Reshape(7,7,128), Conv2DTranspose(64, 5, strides=2, padding='same'), Conv2DTranspose(1, 5, strides=2, padding='same', activation='tanh')

B Sequential model with Dense(100), Reshape(10,10,1), Conv2DTranspose(1, 3, strides=3, padding='valid', activation='relu')

C Sequential model with Dense(28*28), Reshape(28,28,1), Conv2D(64, 3, padding='same'), Conv2D(1, 3, padding='same', activation='sigmoid')

D Sequential model with Dense(784), Reshape(28,28,1), Conv2DTranspose(1, 3, strides=1, padding='same', activation='sigmoid')

💡 Hint

The generator uses Conv2DTranspose layers to upscale from a small feature map to 28x28.

Option A correctly starts from a dense layer to a small 7x7 feature map, then uses two Conv2DTranspose layers with stride 2 to upscale to 28x28, outputting 1 channel with tanh activation.

❓ Hyperparameter

advanced

Best learning rate for GAN training stability

Which learning rate is most commonly recommended to start training a simple GAN with Adam optimizer for stable convergence?

A 0.1

B 0.5

C 0.01

D 0.0002

💡 Hint

GANs usually require small learning rates to avoid instability.

A learning rate of 0.0002 with Adam optimizer is a common default for GANs to balance learning speed and stability.

❓ Metrics

advanced

Interpreting GAN discriminator loss during training

If the discriminator loss is close to 0.0 during GAN training, what does it indicate about the discriminator's performance?

A The discriminator is perfectly guessing real vs fake images.

B The discriminator is failing to distinguish real from fake images.

C The discriminator is outputting random guesses.

D The discriminator is overfitting and memorizing the training data.

💡 Hint

Low loss means the model is confident and correct.

A discriminator loss near 0 means it is confidently and correctly classifying real and fake images, indicating strong performance.

🔧 Debug

expert

Identifying cause of mode collapse in GAN training

During GAN training, the generator produces very similar images repeatedly (mode collapse). Which cause is most likely responsible?

A The batch size is too large, causing noisy gradients.

B The generator learning rate is too low, causing slow updates.

C The discriminator is too weak and cannot provide useful feedback.

D The generator is over-regularized with dropout layers.

💡 Hint

Mode collapse often happens when the discriminator fails to challenge the generator.

If the discriminator is too weak, it cannot push the generator to explore diverse outputs, causing mode collapse.

Practice

1. In a simple GAN, what is the main role of the generator?

easy

A. To classify data as real or fake

B. To create fake data that looks real

C. To optimize the loss function of the discriminator

D. To preprocess the input data

Solution

1. Step 1: Understand the generator's purpose

   The generator's job is to produce fake data samples that resemble real data.

2. Step 2: Differentiate from discriminator's role

   The discriminator's role is to distinguish real from fake data, not the generator's.

3. Final Answer:

   To create fake data that looks real -> Option B

4. Quick Check:

   Generator = create fake data [OK]

Hint: Generator makes fake data to fool discriminator [OK]

Common Mistakes:

• Confusing generator with discriminator
• Thinking generator classifies data
• Assuming generator preprocesses data

2. Which of the following is the correct way to define the input noise vector for the generator in TensorFlow?

easy

A. noise = tf.random.normal([batch_size, noise_dim])

B. noise = tf.zeros([batch_size, noise_dim])

C. noise = tf.ones([batch_size, noise_dim])

D. noise = tf.random.uniform([noise_dim])

Solution

1. Step 1: Identify typical noise input shape

   The noise vector usually has shape [batch_size, noise_dim] to generate a batch of samples.

2. Step 2: Check correct TensorFlow function

   tf.random.normal generates random normal noise, commonly used for GAN input.

3. Final Answer:

   noise = tf.random.normal([batch_size, noise_dim]) -> Option A

4. Quick Check:

   Use tf.random.normal with batch size and noise dim [OK]

Hint: Use tf.random.normal with batch and noise dims [OK]

Common Mistakes:

• Using tf.zeros or tf.ones which produce constant noise
• Missing batch dimension in noise shape
• Using tf.random.uniform without batch dimension

3. Given this TensorFlow code snippet for a GAN training step, what will be the shape of fake_output?

noise = tf.random.normal([32, 100])
fake_images = generator(noise)
fake_output = discriminator(fake_images)

medium

A. [32, 1]

B. [100, 32]

C. [32, 100]

D. [1, 32]

Solution

1. Step 1: Analyze input batch size

   The noise input has batch size 32, so generator outputs 32 fake images.

2. Step 2: Understand discriminator output shape

   The discriminator outputs a prediction per image, usually shape [batch_size, 1].

3. Final Answer:

   [32, 1] -> Option A

4. Quick Check:

   Discriminator output shape = [batch_size, 1] [OK]

Hint: Discriminator outputs one score per input image [OK]

Common Mistakes:

• Confusing noise dimension with output shape
• Swapping batch size and feature dimensions
• Assuming discriminator outputs full image shape

4. You have this error during GAN training: ValueError: Shapes (32, 1) and (64, 1) are incompatible. What is the most likely cause?

medium

A. Discriminator model is missing an output layer

B. Generator output shape is incorrect

C. Mismatch between batch sizes of real and fake data during training

D. Noise vector dimension is too large

Solution

1. Step 1: Understand the error message

   The error shows two tensors with batch sizes 32 and 64 being compared or combined.

2. Step 2: Identify cause in GAN training

   Real and fake batches must have the same batch size when computing loss or concatenating.

3. Final Answer:

   Mismatch between batch sizes of real and fake data during training -> Option C

4. Quick Check:

   Batch size mismatch causes shape errors [OK]

Hint: Ensure real and fake batches have same size [OK]

Common Mistakes:

• Ignoring batch size mismatch
• Blaming generator output shape without checking batch
• Assuming model layers missing cause shape error

5. You want to improve your simple GAN's training stability. Which change is best to try first?

hard

A. Train discriminator twice as often as generator

B. Increase noise vector dimension to 1000

C. Remove batch normalization layers from generator

D. Use label smoothing on discriminator labels

Solution

1. Step 1: Understand training stability issues

   GANs often suffer from unstable training causing oscillations or mode collapse.

2. Step 2: Evaluate common techniques

   Label smoothing (e.g., using 0.9 instead of 1 for real labels) helps discriminator not be too confident, improving stability.

3. Final Answer:

   Use label smoothing on discriminator labels -> Option D

4. Quick Check:

   Label smoothing improves GAN training stability [OK]

Hint: Try label smoothing first to stabilize GAN training [OK]

Common Mistakes:

• Increasing noise dimension too much adds complexity
• Removing batch norm can hurt generator learning
• Training discriminator more often can cause imbalance