Simple GAN implementation in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a GAN in simple terms?

A GAN, or Generative Adversarial Network, is a pair of computer programs that learn together. One tries to create fake data, and the other tries to spot if the data is fake or real. They improve by competing with each other.

beginner

What are the two main parts of a GAN?

The two main parts are the Generator, which makes fake data, and the Discriminator, which checks if data is real or fake.

beginner

Why do we use random noise as input to the Generator?

Random noise is like a blank canvas. The Generator learns to turn this noise into data that looks real, like images or sounds.

beginner

What does the Discriminator output represent?

The Discriminator outputs a number between 0 and 1 that shows how sure it is that the input data is real (close to 1) or fake (close to 0).

beginner

What is the goal of training a GAN?

The goal is for the Generator to make data so real that the Discriminator cannot tell it is fake, and for the Discriminator to get better at spotting fakes. This makes both improve.

What does the Generator in a GAN do?

A Creates fake data from random noise

B Classifies data as real or fake

C Prepares training labels

D Evaluates model accuracy

The Generator creates fake data starting from random noise to fool the Discriminator.

What is the Discriminator's main task?

A Generate new data samples

B Detect if data is real or fake

C Add noise to data

D Reduce model loss

The Discriminator learns to tell if data is real or generated (fake).

Which loss function is commonly used to train GANs?

A Cross-Entropy Loss

B Mean Squared Error

C Hinge Loss

D L1 Loss

Cross-Entropy Loss is commonly used to train both Generator and Discriminator in GANs.

What input does the Generator receive?

A Labels

B Real images

C Random noise vectors

D Discriminator outputs

The Generator takes random noise vectors as input to create fake data.

During GAN training, what happens when the Discriminator gets too good?

A Discriminator generates data

B Training stops

C Generator improves faster

D Generator struggles to fool Discriminator

If the Discriminator is too good, the Generator finds it hard to fool it, which can slow training.

Explain how the Generator and Discriminator work together in a GAN.

Think of a forger and a detective improving their skills.

Describe the training process of a simple GAN step-by-step.

Focus on alternating training of two models.

Practice

1. In a simple GAN, what is the main role of the generator?

easy

A. To classify data as real or fake

B. To create fake data that looks real

C. To optimize the loss function of the discriminator

D. To preprocess the input data

Solution

1. Step 1: Understand the generator's purpose

   The generator's job is to produce fake data samples that resemble real data.

2. Step 2: Differentiate from discriminator's role

   The discriminator's role is to distinguish real from fake data, not the generator's.

3. Final Answer:

   To create fake data that looks real -> Option B

4. Quick Check:

   Generator = create fake data [OK]

Hint: Generator makes fake data to fool discriminator [OK]

Common Mistakes:

• Confusing generator with discriminator
• Thinking generator classifies data
• Assuming generator preprocesses data

2. Which of the following is the correct way to define the input noise vector for the generator in TensorFlow?

easy

A. noise = tf.random.normal([batch_size, noise_dim])

B. noise = tf.zeros([batch_size, noise_dim])

C. noise = tf.ones([batch_size, noise_dim])

D. noise = tf.random.uniform([noise_dim])

Solution

1. Step 1: Identify typical noise input shape

   The noise vector usually has shape [batch_size, noise_dim] to generate a batch of samples.

2. Step 2: Check correct TensorFlow function

   tf.random.normal generates random normal noise, commonly used for GAN input.

3. Final Answer:

   noise = tf.random.normal([batch_size, noise_dim]) -> Option A

4. Quick Check:

   Use tf.random.normal with batch size and noise dim [OK]

Hint: Use tf.random.normal with batch and noise dims [OK]

Common Mistakes:

• Using tf.zeros or tf.ones which produce constant noise
• Missing batch dimension in noise shape
• Using tf.random.uniform without batch dimension

3. Given this TensorFlow code snippet for a GAN training step, what will be the shape of fake_output?

noise = tf.random.normal([32, 100])
fake_images = generator(noise)
fake_output = discriminator(fake_images)

medium

A. [32, 1]

B. [100, 32]

C. [32, 100]

D. [1, 32]

Solution

1. Step 1: Analyze input batch size

   The noise input has batch size 32, so generator outputs 32 fake images.

2. Step 2: Understand discriminator output shape

   The discriminator outputs a prediction per image, usually shape [batch_size, 1].

3. Final Answer:

   [32, 1] -> Option A

4. Quick Check:

   Discriminator output shape = [batch_size, 1] [OK]

Hint: Discriminator outputs one score per input image [OK]

Common Mistakes:

• Confusing noise dimension with output shape
• Swapping batch size and feature dimensions
• Assuming discriminator outputs full image shape

4. You have this error during GAN training: ValueError: Shapes (32, 1) and (64, 1) are incompatible. What is the most likely cause?

medium

A. Discriminator model is missing an output layer

B. Generator output shape is incorrect

C. Mismatch between batch sizes of real and fake data during training

D. Noise vector dimension is too large

Solution

1. Step 1: Understand the error message

   The error shows two tensors with batch sizes 32 and 64 being compared or combined.

2. Step 2: Identify cause in GAN training

   Real and fake batches must have the same batch size when computing loss or concatenating.

3. Final Answer:

   Mismatch between batch sizes of real and fake data during training -> Option C

4. Quick Check:

   Batch size mismatch causes shape errors [OK]

Hint: Ensure real and fake batches have same size [OK]

Common Mistakes:

• Ignoring batch size mismatch
• Blaming generator output shape without checking batch
• Assuming model layers missing cause shape error

5. You want to improve your simple GAN's training stability. Which change is best to try first?

hard

A. Train discriminator twice as often as generator

B. Increase noise vector dimension to 1000

C. Remove batch normalization layers from generator

D. Use label smoothing on discriminator labels

Solution

1. Step 1: Understand training stability issues

   GANs often suffer from unstable training causing oscillations or mode collapse.

2. Step 2: Evaluate common techniques

   Label smoothing (e.g., using 0.9 instead of 1 for real labels) helps discriminator not be too confident, improving stability.

3. Final Answer:

   Use label smoothing on discriminator labels -> Option D

4. Quick Check:

   Label smoothing improves GAN training stability [OK]

Hint: Try label smoothing first to stabilize GAN training [OK]

Common Mistakes:

• Increasing noise dimension too much adds complexity
• Removing batch norm can hurt generator learning
• Training discriminator more often can cause imbalance