Why generative models create visual content in Computer Vision - Why Metrics Matter

Metrics & Evaluation - Why generative models create visual content

Which metric matters for this concept and WHY

For generative models creating visual content, the key metrics are Inception Score (IS) and Fréchet Inception Distance (FID). These metrics measure how realistic and diverse the generated images are. IS checks if images look like real objects and are varied. FID compares generated images to real ones to see how close they are in quality. These matter because visual content must look believable and not repetitive.

Confusion matrix or equivalent visualization (ASCII)

Generative models don't use confusion matrices like classifiers. Instead, we visualize results with example images and metric scores.

    Real Images:    [Cat, Dog, Car, Tree]
    Generated Images: [Cat-like, Dog-like, Car-like, Tree-like]

    Inception Score: 7.5 (higher is better, max ~10)
    FID Score: 25.0 (lower is better, 0 is perfect)
    

This shows how close generated images are to real ones in quality and variety.

Precision vs Recall (or equivalent tradeoff) with concrete examples

In generative visuals, precision means how many generated images look real and sharp. Recall means how many different types of images the model can create.

Example: A model that only creates perfect cats has high precision but low recall (no dogs or cars). Another model creates many types but some look blurry, so recall is high but precision is low.

Good models balance both: images look real and cover many categories.

What "good" vs "bad" metric values look like for this use case

Good values:

• Inception Score (IS) above 7 means images are realistic and varied.
• FID below 30 means generated images are close to real images.

Bad values:

• IS below 3 means images are poor quality or repetitive.
• FID above 100 means images look very different from real ones.

Metrics pitfalls (accuracy paradox, data leakage, overfitting indicators)

Common pitfalls include:

• Mode collapse: Model generates only a few images repeatedly, causing low diversity but possibly high precision.
• Overfitting: Model memorizes training images, so metrics look good but new images are not creative.
• Misleading IS: High IS can happen if images are sharp but unrealistic.
• Data leakage: Using test images in training can falsely improve metrics.

Self-check: Your model has Inception Score 8.0 but FID 120. Is it good?

No, this means the model creates sharp images (high IS) but they are very different from real images (high FID). The images might look unrealistic or have artifacts. The model needs improvement to generate more realistic visuals.

Key Result

Inception Score and FID are key to measure realism and diversity of generated visual content.

Practice

1. Why do generative models create visual content in computer vision?

easy

A. To learn patterns from images and create new, similar images

B. To classify images into categories

C. To detect edges in images

D. To compress images for storage

Solution

1. Step 1: Understand the purpose of generative models

   Generative models learn from many example images to understand patterns and features.

2. Step 2: Identify the output of generative models

   They use learned patterns to create new images that look similar but are not copies.

3. Final Answer:

   To learn patterns from images and create new, similar images -> Option A

4. Quick Check:

   Generative models create new images = A [OK]

Hint: Generative means creating new data, not just analyzing [OK]

Common Mistakes:

• Confusing generative models with classifiers
• Thinking generative models only compress images
• Assuming generative models detect edges only

2. Which of the following is the correct way to describe a generative model's output?

easy

A. A label indicating the image category

B. A compressed version of the input image

C. A new image created from learned patterns

D. A list of detected edges in the image

Solution

1. Step 1: Identify what generative models produce

   Generative models output new images, not labels or compressed data.

2. Step 2: Match output type to options

   A new image created from learned patterns describes new images created from learned patterns, which fits generative models.

3. Final Answer:

   A new image created from learned patterns -> Option C

4. Quick Check:

   Generative output = new images [OK]

Hint: Generative output is new data, not labels or compression [OK]

Common Mistakes:

• Choosing classification labels as output
• Confusing compression with generation
• Selecting edge detection results

3. Consider this Python code snippet using a generative model:

import numpy as np

# Fake model that generates images
class SimpleGenerator:
    def generate(self, seed):
        np.random.seed(seed)
        return np.random.rand(2, 2)

gen = SimpleGenerator()
image1 = gen.generate(42)
image2 = gen.generate(42)
print(np.array_equal(image1, image2))

What will be the output of this code?

medium

A. True

B. None

C. An error because of missing imports

D. False

Solution

1. Step 1: Understand the effect of setting the random seed

   Setting the seed to the same value before generating random numbers produces the same output.

2. Step 2: Compare the two generated images

   Both calls use seed 42, so image1 and image2 are identical, making np.array_equal return True.

3. Final Answer:

   True -> Option A

4. Quick Check:

   Same seed = same output = True [OK]

Hint: Same random seed means identical outputs [OK]

Common Mistakes:

• Assuming different outputs without seed reset
• Thinking np.array_equal returns arrays
• Ignoring seed effect on randomness

4. The following code tries to generate new images using a generative model but always produces the same output:

class Generator:
    def __init__(self):
        self.counter = 0
    def generate(self):
        return self.counter + 1

model = Generator()
print(model.generate())
print(model.generate())

What is the main issue causing the output to be incorrect for generating new images?

medium

A. The class is missing an import statement

B. The generate method does not update the counter, so output is always 1

C. The generate method returns a string instead of an image

D. The counter variable is not initialized

Solution

1. Step 1: Analyze the generate method behavior

   The method returns self.counter + 1 but does not update self.counter, so it always returns 1.

2. Step 2: Understand why this is a problem for generation

   Without updating the counter, the model cannot produce different outputs over calls, failing to generate new images.

3. Final Answer:

   The generate method does not update the counter, so output is always 1 -> Option B

4. Quick Check:

   Counter not updated = same output [OK]

Hint: Update state variables to get new outputs [OK]

Common Mistakes:

• Thinking missing imports prevent the code from running
• Assuming generate returns images automatically
• Ignoring that counter is initialized but not changed

5. You want to use a generative model to create new images for training a self-driving car system. Which of these is the best reason to use generative models for this task?

hard

A. To detect obstacles in real-time video

B. To label images automatically with road signs

C. To compress the training images for faster loading

D. To generate many varied images that help the system learn different driving conditions

Solution

1. Step 1: Identify the goal of using generative models in training

   Generative models create new images that add variety and cover different scenarios for better learning.

2. Step 2: Match the goal to the options

   To generate many varied images that help the system learn different driving conditions explains generating varied images to improve training, which fits the use case.

3. Final Answer:

   To generate many varied images that help the system learn different driving conditions -> Option D

4. Quick Check:

   Generative models create varied training data = A [OK]

Hint: Use generative models to expand training data variety [OK]

Common Mistakes:

• Confusing generation with labeling or detection
• Thinking generative models compress data
• Assuming generative models work in real-time detection