Simple GAN implementation in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Simple GAN implementation

Which metric matters for this concept and WHY

For a Simple GAN (Generative Adversarial Network), the key metrics are Discriminator Loss and Generator Loss. These losses show how well the discriminator can tell real from fake data and how well the generator can fool the discriminator. Lower discriminator loss means it is good at spotting fakes. Lower generator loss means it is good at creating realistic fake data. We also look at visual quality of generated samples because GANs create images or data, so numbers alone don't tell the full story.

Confusion matrix or equivalent visualization (ASCII)

GANs don't use a classic confusion matrix because they generate data, not classify fixed labels. But we can think of the discriminator's task as a binary classifier:

          Predicted Real | Predicted Fake
Actual Real    TP       |     FN
Actual Fake    FP       |     TN

Here, TP means real images correctly identified, FP means fake images wrongly called real, FN means real images wrongly called fake, and TN means fake images correctly identified. The discriminator tries to maximize TP and TN while minimizing FP and FN.

Precision vs Recall (or equivalent tradeoff) with concrete examples

In GANs, the tradeoff is between the discriminator and generator:

• Discriminator too strong: It easily spots fakes, so generator struggles and produces poor images.
• Generator too strong: It fools discriminator easily, but discriminator can't learn to improve.

Good training balances both. For example, if the generator creates blurry images, discriminator loss is low (easy to spot), generator loss high. If generator creates sharp but unrealistic images, discriminator loss might be high but generator loss low. The goal is to reach a balance where generated images look real and discriminator is challenged.

What "good" vs "bad" metric values look like for this use case

Good:

• Discriminator loss and generator loss both stabilize around similar values.
• Generated images look clear and realistic.
• Neither loss is near zero or very high for long.

Bad:

• Discriminator loss near zero and generator loss very high (discriminator too strong).
• Generator loss near zero and discriminator loss very high (generator too strong, discriminator fails).
• Losses fluctuate wildly without stabilizing.
• Generated images are noisy, blurry, or obviously fake.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

• Accuracy paradox: Accuracy is not used because GANs don't classify fixed labels.
• Data leakage: Using test data in training can make discriminator too good, hiding generator problems.
• Overfitting: Discriminator memorizing training data leads to poor generalization and bad generator feedback.
• Mode collapse: Generator produces limited variety of outputs, metrics may look good but diversity is lost.

Your model has 98% accuracy but 12% recall on fraud. Is it good?

This question is about fraud detection, not GANs, but it teaches an important lesson. High accuracy with very low recall means the model misses most fraud cases. For fraud detection, missing fraud is very bad. So despite 98% accuracy, the model is not good for production because it fails to catch fraud. Similarly, in GANs, a single metric is not enough; we must check multiple losses and output quality.

Key Result

In Simple GANs, balanced discriminator and generator losses with realistic generated samples indicate good performance.

Practice

1. In a simple GAN, what is the main role of the generator?

easy

A. To classify data as real or fake

B. To create fake data that looks real

C. To optimize the loss function of the discriminator

D. To preprocess the input data

Solution

1. Step 1: Understand the generator's purpose

   The generator's job is to produce fake data samples that resemble real data.

2. Step 2: Differentiate from discriminator's role

   The discriminator's role is to distinguish real from fake data, not the generator's.

3. Final Answer:

   To create fake data that looks real -> Option B

4. Quick Check:

   Generator = create fake data [OK]

Hint: Generator makes fake data to fool discriminator [OK]

Common Mistakes:

• Confusing generator with discriminator
• Thinking generator classifies data
• Assuming generator preprocesses data

2. Which of the following is the correct way to define the input noise vector for the generator in TensorFlow?

easy

A. noise = tf.random.normal([batch_size, noise_dim])

B. noise = tf.zeros([batch_size, noise_dim])

C. noise = tf.ones([batch_size, noise_dim])

D. noise = tf.random.uniform([noise_dim])

Solution

1. Step 1: Identify typical noise input shape

   The noise vector usually has shape [batch_size, noise_dim] to generate a batch of samples.

2. Step 2: Check correct TensorFlow function

   tf.random.normal generates random normal noise, commonly used for GAN input.

3. Final Answer:

   noise = tf.random.normal([batch_size, noise_dim]) -> Option A

4. Quick Check:

   Use tf.random.normal with batch size and noise dim [OK]

Hint: Use tf.random.normal with batch and noise dims [OK]

Common Mistakes:

• Using tf.zeros or tf.ones which produce constant noise
• Missing batch dimension in noise shape
• Using tf.random.uniform without batch dimension

3. Given this TensorFlow code snippet for a GAN training step, what will be the shape of fake_output?

noise = tf.random.normal([32, 100])
fake_images = generator(noise)
fake_output = discriminator(fake_images)

medium

A. [32, 1]

B. [100, 32]

C. [32, 100]

D. [1, 32]

Solution

1. Step 1: Analyze input batch size

   The noise input has batch size 32, so generator outputs 32 fake images.

2. Step 2: Understand discriminator output shape

   The discriminator outputs a prediction per image, usually shape [batch_size, 1].

3. Final Answer:

   [32, 1] -> Option A

4. Quick Check:

   Discriminator output shape = [batch_size, 1] [OK]

Hint: Discriminator outputs one score per input image [OK]

Common Mistakes:

• Confusing noise dimension with output shape
• Swapping batch size and feature dimensions
• Assuming discriminator outputs full image shape

4. You have this error during GAN training: ValueError: Shapes (32, 1) and (64, 1) are incompatible. What is the most likely cause?

medium

A. Discriminator model is missing an output layer

B. Generator output shape is incorrect

C. Mismatch between batch sizes of real and fake data during training

D. Noise vector dimension is too large

Solution

1. Step 1: Understand the error message

   The error shows two tensors with batch sizes 32 and 64 being compared or combined.

2. Step 2: Identify cause in GAN training

   Real and fake batches must have the same batch size when computing loss or concatenating.

3. Final Answer:

   Mismatch between batch sizes of real and fake data during training -> Option C

4. Quick Check:

   Batch size mismatch causes shape errors [OK]

Hint: Ensure real and fake batches have same size [OK]

Common Mistakes:

• Ignoring batch size mismatch
• Blaming generator output shape without checking batch
• Assuming model layers missing cause shape error

5. You want to improve your simple GAN's training stability. Which change is best to try first?

hard

A. Train discriminator twice as often as generator

B. Increase noise vector dimension to 1000

C. Remove batch normalization layers from generator

D. Use label smoothing on discriminator labels

Solution

1. Step 1: Understand training stability issues

   GANs often suffer from unstable training causing oscillations or mode collapse.

2. Step 2: Evaluate common techniques

   Label smoothing (e.g., using 0.9 instead of 1 for real labels) helps discriminator not be too confident, improving stability.

3. Final Answer:

   Use label smoothing on discriminator labels -> Option D

4. Quick Check:

   Label smoothing improves GAN training stability [OK]

Hint: Try label smoothing first to stabilize GAN training [OK]

Common Mistakes:

• Increasing noise dimension too much adds complexity
• Removing batch norm can hurt generator learning
• Training discriminator more often can cause imbalance