GAN concept (generator and discriminator) in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - GAN concept (generator and discriminator)

Which metric matters for GAN concept (generator and discriminator) and WHY

In GANs, we want the generator to create data that looks real, and the discriminator to tell real from fake. The key metrics are discriminator loss and generator loss. These losses show how well each part is doing its job. Lower loss means better performance. We also look at Inception Score or Fréchet Inception Distance (FID) to measure how real and diverse the generated images are. These metrics help us know if the generator is improving in making realistic data.

Confusion matrix or equivalent visualization

GANs don't use a classic confusion matrix because the discriminator outputs probabilities, not fixed classes. But we can think of it like this:

    Real images:    Discriminator says Real (True Positive, TP)
                    Discriminator says Fake (False Negative, FN)

    Fake images:    Discriminator says Fake (True Negative, TN)
                    Discriminator says Real (False Positive, FP)
    

We want the discriminator to have high TP and TN, and low FP and FN. But if the discriminator is too good, the generator struggles. So the balance is key.

Precision vs Recall tradeoff with concrete examples

In GANs, the discriminator's precision means how often it correctly labels fake images as fake (avoiding false alarms). Recall means how often it correctly identifies real images as real (not missing real data).

If the discriminator has very high precision but low recall, it rarely mistakes fake for real but may miss some real images, making the generator's job easier.

If recall is high but precision is low, the discriminator catches most real images but often mistakes fake images as real, making the generator less challenged.

Good GAN training balances these so the generator improves and the discriminator stays accurate but not perfect.

What "good" vs "bad" metric values look like for GAN concept

Good:

• Discriminator loss and generator loss both decrease and stabilize over time.
• Inception Score increases, showing more realistic and varied images.
• FID score decreases, meaning generated images are closer to real images.
• Discriminator accuracy around 50%, showing it can't easily tell real from fake (ideal balance).

Bad:

• Discriminator loss very low and generator loss very high: discriminator too strong, generator fails.
• Generator loss very low but discriminator loss very high: generator is cheating or discriminator is weak.
• Inception Score low and FID high: generated images look fake or lack variety.
• Discriminator accuracy near 100%: no learning balance, generator not improving.

Metrics pitfalls

• Mode collapse: Generator produces limited variety. Losses may look good but output is poor.
• Overfitting discriminator: Discriminator memorizes training data, causing poor generator learning.
• Ignoring diversity: Good loss values but generated data lacks variety, not captured by simple loss metrics.
• Misinterpreting accuracy: High discriminator accuracy is not always good; it may mean generator is weak.
• Data leakage: Using test data in training can give misleading metrics.

Self-check question

Your GAN's discriminator accuracy is 98%, but the generator loss is very high and generated images look poor. Is this model good for production? Why or why not?

Answer: No, this is not good. The high discriminator accuracy means it easily tells real from fake, so the generator is not learning well. The high generator loss and poor images confirm this. The model needs better balance to improve generator quality.

Key Result

In GANs, balanced generator and discriminator losses plus image quality scores like FID best show model progress.

Practice

1.

What is the main role of the generator in a GAN?

easy

A. To classify data as real or fake

B. To create fake data that looks real

C. To label data for supervised learning

D. To reduce the size of the dataset

Solution

1. Step 1: Understand the generator's purpose

   The generator's job is to produce new data that mimics real data.

2. Step 2: Differentiate from discriminator

   The discriminator checks data authenticity, but the generator creates fake data.

3. Final Answer:

   To create fake data that looks real -> Option B

4. Quick Check:

   Generator = create fake data [OK]

Hint: Generator makes fake data, discriminator checks it [OK]

Common Mistakes:

• Confusing generator with discriminator
• Thinking generator labels data
• Assuming generator reduces data size

2.

Which of the following is the correct way to define a simple generator model in TensorFlow Keras?

def make_generator_model():
    model = tf.keras.Sequential([
        tf.keras.layers.Dense(7*7*256, use_bias=False, input_shape=(100,)),
        tf.keras.layers.BatchNormalization(),
        tf.keras.layers.LeakyReLU(),
        tf.keras.layers.Reshape((7, 7, 256)),
        tf.keras.layers.Conv2DTranspose(128, 5, strides=1, padding='same', use_bias=False),
        tf.keras.layers.BatchNormalization(),
        tf.keras.layers.LeakyReLU(),
        tf.keras.layers.Conv2DTranspose(1, 5, strides=2, padding='same', activation='tanh')
    ])
    return model

easy

A. The model is missing an input layer

B. The activation function 'tanh' is incorrect for generator output

C. The code correctly defines a generator model

D. The Dense layer should have bias enabled

Solution

1. Step 1: Check model layers and input

   The model starts with a Dense layer with input_shape=(100,), which is correct for noise input.

2. Step 2: Verify output activation and structure

   The last Conv2DTranspose uses 'tanh' activation, common for generator output scaled between -1 and 1.

3. Final Answer:

   The code correctly defines a generator model -> Option C

4. Quick Check:

   Generator model code is correct [OK]

Hint: Generator output often uses tanh activation [OK]

Common Mistakes:

• Forgetting input_shape in first layer
• Using wrong activation for output layer
• Enabling bias unnecessarily in Dense layer

3.

Given the following code snippet for a discriminator model, what will be the output shape after the last Conv2D layer?

model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(64, 5, strides=2, padding='same', input_shape=(28, 28, 1)),
    tf.keras.layers.LeakyReLU(),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Conv2D(128, 5, strides=2, padding='same'),
    tf.keras.layers.LeakyReLU(),
    tf.keras.layers.Dropout(0.3),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(1)
])

medium

A. (7, 7, 128)

B. (14, 14, 128)

C. (28, 28, 64)

D. (1, 1, 1)

Solution

1. Step 1: Calculate output after first Conv2D

   Input is (28,28,1), stride=2, padding='same' halves spatial dims: (14,14,64).

2. Step 2: Calculate output after second Conv2D

   Again stride=2, padding='same' halves spatial dims: (7,7,128).

3. Final Answer:

   (7, 7, 128) -> Option A

4. Quick Check:

   Conv2D output shape = (7,7,128) [OK]

Hint: Stride 2 halves spatial dimensions with 'same' padding [OK]

Common Mistakes:

• Forgetting padding='same' keeps output size rounded up
• Misapplying stride effect on dimensions
• Confusing number of filters with spatial size

4.

Identify the error in this GAN training step code snippet:

noise = tf.random.normal([batch_size, noise_dim])
generated_images = generator(noise, training=True)
real_output = discriminator(real_images, training=True)
fake_output = discriminator(generated_images, training=False)

loss = generator_loss(fake_output) + discriminator_loss(real_output, fake_output)

medium

A. Discriminator should be in training mode for fake images

B. Generator should not use training=True during generation

C. Noise shape is incorrect for generator input

D. Loss calculation should not combine generator and discriminator losses

Solution

1. Step 1: Check discriminator training mode

   Discriminator must be in training mode for both real and fake images to update weights properly.

2. Step 2: Identify incorrect training flag

   Here, discriminator(fake images) uses training=False, which disables training behavior like dropout.

3. Final Answer:

   Discriminator should be in training mode for fake images -> Option A

4. Quick Check:

   Discriminator training mode = True for all inputs [OK]

Hint: Discriminator trains on both real and fake data [OK]

Common Mistakes:

• Setting discriminator training=False on fake data
• Confusing generator training flag
• Incorrect noise shape assumptions

5.

You want to improve a GAN to generate clearer images. Which combined approach is best?

1. Increase generator depth and add batch normalization
2. Use a weaker discriminator to avoid overpowering generator
3. Train discriminator more times per generator update
4. Remove dropout layers from discriminator

hard

A. Options 2 and 4 together are best

B. Options 1 and 2 together are best

C. Only option 1 is best

D. Options 1 and 3 together are best

Solution

1. Step 1: Evaluate option 1

   Increasing generator depth and adding batch normalization helps create clearer, stable images.

2. Step 2: Evaluate option 3

   Training discriminator more per generator update helps discriminator learn better, pushing generator to improve.

3. Step 3: Assess options 2 and 4

   Weaker discriminator or removing dropout can reduce training quality and cause poor results.

4. Final Answer:

   Options 1 and 3 together are best -> Option D

5. Quick Check:

   Improve generator + stronger discriminator training = better images [OK]

Hint: Stronger generator + more discriminator training = clearer images [OK]

Common Mistakes:

• Weakening discriminator hurts GAN balance
• Removing dropout may cause overfitting
• Ignoring batch normalization benefits