Image generation basics in TensorFlow - Model Metrics & Evaluation

Metrics & Evaluation - Image generation basics

Which metric matters for Image Generation and WHY

In image generation, we want to know how close the generated images are to real ones. Common metrics include Inception Score (IS) and Fréchet Inception Distance (FID). IS measures how clear and diverse the images are. FID compares the statistics of generated images to real images, showing how similar they look overall. Lower FID and higher IS mean better image quality and variety.

Confusion Matrix or Equivalent Visualization

Image generation does not use a confusion matrix like classification. Instead, we compare sets of images. For example, FID uses features from a neural network to measure distance between real and generated image groups.

Real images features: mean = m_r, covariance = C_r
Generated images features: mean = m_g, covariance = C_g

FID = ||m_r - m_g||^2 + Tr(C_r + C_g - 2 * sqrt(C_r * C_g))

Lower FID means generated images are closer to real ones.
    

Precision vs Recall Tradeoff with Examples

In image generation, precision means how realistic the generated images are (quality). Recall means how many different types of images the model can create (diversity).

For example, a model with high precision but low recall makes very clear images but only a few types. A model with high recall but low precision makes many different images but some look blurry or wrong.

Good image generation balances both: clear images that cover many styles or objects.

What "Good" vs "Bad" Metric Values Look Like

Good: FID below 30 and Inception Score above 8 usually mean the model generates realistic and diverse images.

Bad: FID above 100 or Inception Score below 3 means images are blurry, unrealistic, or very similar to each other.

These numbers depend on the dataset and task but give a general idea.

Common Metrics Pitfalls

• Overfitting: Model memorizes training images, so metrics look good but new images are poor.
• Mode collapse: Model generates only a few types of images, hurting recall.
• Misleading scores: Inception Score can be high if images are clear but all look the same.
• Data leakage: Using test images in training can falsely improve metrics.

Self Check

Your image generator has an Inception Score of 9 but a very high FID of 120. Is it good?

Answer: No. The high Inception Score means images look clear, but the very high FID shows they are not similar to real images overall. The model might produce sharp but unrealistic images or lack diversity. You should improve the model to lower FID and keep IS high.

Key Result

In image generation, balance between image quality (precision) and diversity (recall) is measured by Inception Score and Fréchet Inception Distance.

Practice

1. What is the main role of a generator model in image generation using TensorFlow?

easy

A. To classify images into categories

B. To label images with tags

C. To reduce the size of images

D. To create new images from random noise inputs

Solution

1. Step 1: Understand the generator's purpose

   The generator model takes random noise as input and learns to produce new images that look like real data.

2. Step 2: Differentiate from other models

   Unlike classifiers or reducers, the generator's job is to create images, not to label or compress them.

3. Final Answer:

   To create new images from random noise inputs -> Option D

4. Quick Check:

   Generator creates images = B [OK]

Hint: Generator turns noise into images step-by-step [OK]

Common Mistakes:

• Confusing generator with classifier
• Thinking generator compresses images
• Assuming generator labels images

2. Which of the following TensorFlow code snippets correctly defines a simple generator model layer that upsamples noise to a 7x7x128 feature map?

easy

A. tf.keras.layers.MaxPooling2D((2,2))

B. tf.keras.layers.Dense(7*7*128, use_bias=False, input_shape=(100,))

C. tf.keras.layers.Conv2D(128, (3,3), activation='relu')

D. tf.keras.layers.Flatten()

Solution

1. Step 1: Identify the layer that creates a 7x7x128 feature map from noise

   The Dense layer with units 7*7*128 reshapes noise vector into a flat feature map for upsampling.

2. Step 2: Check other options

   Conv2D applies convolution but does not reshape noise directly; MaxPooling2D reduces size; Flatten converts to 1D.

3. Final Answer:

   tf.keras.layers.Dense(7*7*128, use_bias=False, input_shape=(100,)) -> Option B

4. Quick Check:

   Dense layer reshapes noise = C [OK]

Hint: Dense layer with units = 7*7*128 reshapes noise [OK]

Common Mistakes:

• Using Conv2D before reshaping noise
• Choosing pooling layers for upsampling
• Flattening noise vector instead of reshaping

3. Given the following TensorFlow code snippet for generating an image, what will be the shape of the output image tensor?

noise = tf.random.normal([1, 100])
generated_image = generator(noise, training=False)

Assume the generator outputs images of shape (28, 28, 1).

medium

A. (1, 28, 28, 1)

B. (28, 28, 1)

C. (100, 28, 28, 1)

D. (1, 100)

Solution

1. Step 1: Understand input noise shape

   The noise tensor has shape (1, 100), where 1 is batch size and 100 is noise dimension.

2. Step 2: Check generator output shape

   The generator outputs images shaped (28, 28, 1) per input sample, so output shape includes batch size: (1, 28, 28, 1).

3. Final Answer:

   (1, 28, 28, 1) -> Option A

4. Quick Check:

   Batch size + image shape = (1, 28, 28, 1) [OK]

Hint: Output shape includes batch size from input noise [OK]

Common Mistakes:

• Ignoring batch size dimension
• Confusing noise dimension with image size
• Assuming output shape matches noise shape

4. You wrote this TensorFlow code to generate images but get an error:

noise = tf.random.normal([100])
generated_image = generator(noise, training=False)

What is the likely cause of the error?

medium

A. Noise input is missing batch dimension

B. Generator model is not compiled

C. Noise tensor has wrong data type

D. Training flag must be True for generation

Solution

1. Step 1: Check noise input shape

   The noise tensor shape is (100,), missing batch dimension which should be (1, 100) or more.

2. Step 2: Understand generator input requirements

   Generator expects batch dimension to process inputs; missing it causes shape mismatch error.

3. Final Answer:

   Noise input is missing batch dimension -> Option A

4. Quick Check:

   Batch dimension required in noise input = A [OK]

Hint: Add batch size dimension to noise input shape [OK]

Common Mistakes:

• Passing 1D noise without batch size
• Assuming training flag affects input shape
• Ignoring model compilation errors

5. You want to generate 5 new images using a trained TensorFlow generator model. Which code snippet correctly generates and displays these images using matplotlib?

hard

A. noise = tf.random.normal([5, 100])
generated_images = generator(noise)
plt.imshow(generated_images, cmap='gray')
plt.show()

B. noise = tf.random.normal([1, 100])
generated_images = generator(noise, training=False)
plt.imshow(generated_images, cmap='gray')
plt.show()

C. noise = tf.random.normal([5, 100])
generated_images = generator(noise, training=False)
for i in range(5):
plt.subplot(1, 5, i+1)
plt.imshow(generated_images[i, :, :, 0], cmap='gray')
plt.axis('off')
plt.show()

D. noise = tf.random.normal([5])
generated_images = generator(noise, training=True)
plt.imshow(generated_images[0], cmap='gray')
plt.show()

Solution

1. Step 1: Generate noise for 5 images

   Noise shape [5, 100] creates batch of 5 inputs for generator.

2. Step 2: Generate images and plot each

   Loop over generated_images to plot each image separately with plt.subplot and plt.imshow.

3. Step 3: Check other options

   noise = tf.random.normal([1, 100]) generated_images = generator(noise, training=False) plt.imshow(generated_images, cmap='gray') plt.show() generates only 1 image; another misses training=False and plots batch incorrectly; another has wrong noise shape and training mode.

4. Final Answer:

   noise = tf.random.normal([5, 100]) generated_images = generator(noise, training=False) for i in range(5): plt.subplot(1, 5, i+1) plt.imshow(generated_images[i, :, :, 0], cmap='gray') plt.axis('off') plt.show() -> Option C

5. Quick Check:

   Batch noise + loop display = A [OK]

Hint: Use batch noise and loop to display images with matplotlib [OK]

Common Mistakes:

• Using wrong noise shape for batch
• Not looping to display each image
• Forgetting training=False during generation