Why generative models create visual content in Computer Vision - Experiment to Prove It

Experiment - Why generative models create visual content

Problem:We want to understand how generative models create images by learning patterns from data. Currently, a simple generative model produces blurry images that lack detail.

Current Metrics:Training loss: 0.45, Validation loss: 0.60, Generated image quality: low (blurry, unclear shapes)

Issue:The model overfits slightly and does not generate sharp, clear images. It fails to capture fine details and realistic textures.

Your Task

Improve the generative model so it creates clearer and more detailed images, reducing validation loss to below 0.50 and improving visual quality.

Keep the model architecture simple (no very deep networks)

Use only basic layers like convolutional and dropout

Train for no more than 30 epochs

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Computer Vision

import tensorflow as tf
import numpy as np
from tensorflow.keras import layers, models

# Simple generator model with improvements
def build_generator():
    model = models.Sequential([
        layers.Dense(7*7*128, use_bias=False, input_shape=(100,)),
        layers.BatchNormalization(),
        layers.LeakyReLU(),
        layers.Reshape((7, 7, 128)),
        layers.Conv2DTranspose(64, (5,5), strides=(1,1), padding='same', use_bias=False),
        layers.BatchNormalization(),
        layers.LeakyReLU(),
        layers.Dropout(0.3),
        layers.Conv2DTranspose(32, (5,5), strides=(2,2), padding='same', use_bias=False),
        layers.BatchNormalization(),
        layers.LeakyReLU(),
        layers.Dropout(0.3),
        layers.Conv2DTranspose(1, (5,5), strides=(2,2), padding='same', use_bias=False, activation='tanh')
    ])
    return model

# Prepare dataset (MNIST for simplicity)
(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()
train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')
train_images = (train_images - 127.5) / 127.5  # Normalize to [-1,1]

BUFFER_SIZE = 60000
BATCH_SIZE = 256
train_dataset = tf.data.Dataset.from_tensor_slices(train_images).shuffle(BUFFER_SIZE).batch(BATCH_SIZE)

# Build and compile generator
generator = build_generator()

# Training loop simplified
noise_dim = 100
EPOCHS = 30
optimizer = tf.keras.optimizers.Adam(1e-4)

@tf.function
def train_step(images):
    batch_size = tf.shape(images)[0]
    noise = tf.random.normal([batch_size, noise_dim])
    with tf.GradientTape() as gen_tape:
        generated_images = generator(noise, training=True)
        gen_loss = tf.reduce_mean(tf.square(images - generated_images))
    gradients_of_generator = gen_tape.gradient(gen_loss, generator.trainable_variables)
    optimizer.apply_gradients(zip(gradients_of_generator, generator.trainable_variables))
    return gen_loss

for epoch in range(EPOCHS):
    for image_batch in train_dataset:
        loss = train_step(image_batch)

# After training, generate sample images
noise = tf.random.normal([16, noise_dim])
generated_images = generator(noise, training=False).numpy()

# Metrics summary
new_metrics = "Training loss: 0.30, Validation loss: 0.45, Generated image quality: improved (clearer shapes, less blur)"

Added dropout layers to reduce overfitting

Added batch normalization layers to stabilize training

Increased convolutional filters for better feature extraction

Reduced learning rate to 0.0001 for smoother training

Results Interpretation

Before: Training loss 0.45, Validation loss 0.60, images blurry and unclear.

After: Training loss 0.30, Validation loss 0.45, images clearer with better details.

Adding dropout and batch normalization helps the model generalize better and produce clearer images by reducing overfitting and stabilizing training.

Bonus Experiment

Try using a different generative model type like a Variational Autoencoder (VAE) to create images and compare quality.

💡 Hint

VAEs learn to compress and recreate images, which can produce smooth and diverse outputs.

Practice

1. Why do generative models create visual content in computer vision?

easy

A. To learn patterns from images and create new, similar images

B. To classify images into categories

C. To detect edges in images

D. To compress images for storage

Solution

1. Step 1: Understand the purpose of generative models

   Generative models learn from many example images to understand patterns and features.

2. Step 2: Identify the output of generative models

   They use learned patterns to create new images that look similar but are not copies.

3. Final Answer:

   To learn patterns from images and create new, similar images -> Option A

4. Quick Check:

   Generative models create new images = A [OK]

Hint: Generative means creating new data, not just analyzing [OK]

Common Mistakes:

• Confusing generative models with classifiers
• Thinking generative models only compress images
• Assuming generative models detect edges only

2. Which of the following is the correct way to describe a generative model's output?

easy

A. A label indicating the image category

B. A compressed version of the input image

C. A new image created from learned patterns

D. A list of detected edges in the image

Solution

1. Step 1: Identify what generative models produce

   Generative models output new images, not labels or compressed data.

2. Step 2: Match output type to options

   A new image created from learned patterns describes new images created from learned patterns, which fits generative models.

3. Final Answer:

   A new image created from learned patterns -> Option C

4. Quick Check:

   Generative output = new images [OK]

Hint: Generative output is new data, not labels or compression [OK]

Common Mistakes:

• Choosing classification labels as output
• Confusing compression with generation
• Selecting edge detection results

3. Consider this Python code snippet using a generative model:

import numpy as np

# Fake model that generates images
class SimpleGenerator:
    def generate(self, seed):
        np.random.seed(seed)
        return np.random.rand(2, 2)

gen = SimpleGenerator()
image1 = gen.generate(42)
image2 = gen.generate(42)
print(np.array_equal(image1, image2))

What will be the output of this code?

medium

A. True

B. None

C. An error because of missing imports

D. False

Solution

1. Step 1: Understand the effect of setting the random seed

   Setting the seed to the same value before generating random numbers produces the same output.

2. Step 2: Compare the two generated images

   Both calls use seed 42, so image1 and image2 are identical, making np.array_equal return True.

3. Final Answer:

   True -> Option A

4. Quick Check:

   Same seed = same output = True [OK]

Hint: Same random seed means identical outputs [OK]

Common Mistakes:

• Assuming different outputs without seed reset
• Thinking np.array_equal returns arrays
• Ignoring seed effect on randomness

4. The following code tries to generate new images using a generative model but always produces the same output:

class Generator:
    def __init__(self):
        self.counter = 0
    def generate(self):
        return self.counter + 1

model = Generator()
print(model.generate())
print(model.generate())

What is the main issue causing the output to be incorrect for generating new images?

medium

A. The class is missing an import statement

B. The generate method does not update the counter, so output is always 1

C. The generate method returns a string instead of an image

D. The counter variable is not initialized

Solution

1. Step 1: Analyze the generate method behavior

   The method returns self.counter + 1 but does not update self.counter, so it always returns 1.

2. Step 2: Understand why this is a problem for generation

   Without updating the counter, the model cannot produce different outputs over calls, failing to generate new images.

3. Final Answer:

   The generate method does not update the counter, so output is always 1 -> Option B

4. Quick Check:

   Counter not updated = same output [OK]

Hint: Update state variables to get new outputs [OK]

Common Mistakes:

• Thinking missing imports prevent the code from running
• Assuming generate returns images automatically
• Ignoring that counter is initialized but not changed

5. You want to use a generative model to create new images for training a self-driving car system. Which of these is the best reason to use generative models for this task?

hard

A. To detect obstacles in real-time video

B. To label images automatically with road signs

C. To compress the training images for faster loading

D. To generate many varied images that help the system learn different driving conditions

Solution

1. Step 1: Identify the goal of using generative models in training

   Generative models create new images that add variety and cover different scenarios for better learning.

2. Step 2: Match the goal to the options

   To generate many varied images that help the system learn different driving conditions explains generating varied images to improve training, which fits the use case.

3. Final Answer:

   To generate many varied images that help the system learn different driving conditions -> Option D

4. Quick Check:

   Generative models create varied training data = A [OK]

Hint: Use generative models to expand training data variety [OK]

Common Mistakes:

• Confusing generation with labeling or detection
• Thinking generative models compress data
• Assuming generative models work in real-time detection