Variational Autoencoder in Computer Vision - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of the KL divergence term in a Variational Autoencoder?

In a Variational Autoencoder (VAE), the loss function includes a KL divergence term. What role does this term play?

A It measures the reconstruction error between input and output images.

B It increases the complexity of the decoder network to improve output quality.

C It forces the encoded latent variables to follow a prior distribution, usually a standard normal distribution.

D It acts as a regularizer to prevent overfitting by dropping neurons randomly.

💡 Hint

Think about how the latent space is shaped in a VAE.

The KL divergence term ensures the latent variables approximate a known prior distribution, enabling smooth sampling and generation.

❓ Predict Output

intermediate

What is the output shape of the latent vector z in this VAE encoder snippet?

Given the following PyTorch encoder code snippet, what is the shape of the latent vector z after sampling?

Computer Vision

import torch
import torch.nn as nn

class Encoder(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 400)
        self.fc_mu = nn.Linear(400, 20)
        self.fc_logvar = nn.Linear(400, 20)

    def forward(self, x):
        h = torch.relu(self.fc1(x))
        mu = self.fc_mu(h)
        logvar = self.fc_logvar(h)
        std = torch.exp(0.5 * logvar)
        eps = torch.randn_like(std)
        z = mu + eps * std
        return z

encoder = Encoder()
x = torch.randn(64, 784)
z = encoder(x)

A (64, 20)

B (20, 64)

C (64, 400)

D (400, 20)

💡 Hint

Check the output size of fc_mu and fc_logvar.

The latent vector z has the same batch size as input (64) and latent dimension size (20).

❓ Model Choice

advanced

Which decoder architecture is best suited for reconstructing 28x28 grayscale images in a VAE?

You want to build a decoder for a VAE that reconstructs 28x28 grayscale images (like MNIST). Which architecture is most appropriate?

A A convolutional neural network with transposed convolutions to upsample from latent space to 28x28.

B A recurrent neural network that generates pixels sequentially.

C A fully connected network that outputs a vector of size 784, reshaped to 28x28.

D A simple linear layer that outputs a single scalar value.

💡 Hint

Consider how spatial information is best preserved and reconstructed.

Transposed convolutions in a CNN decoder help reconstruct spatial structure in images better than fully connected layers alone.

❓ Metrics

advanced

Which metric combination correctly evaluates a trained VAE on image reconstruction?

After training a VAE on images, which combination of metrics best evaluates its performance?

A F1 score for reconstruction and BLEU score for latent space.

B Mean Squared Error (MSE) for reconstruction quality and KL divergence for latent distribution regularization.

C Precision and Recall for latent space clustering.

D Accuracy for classification and Cross-Entropy loss for reconstruction.

💡 Hint

Think about what the VAE tries to minimize during training.

MSE measures how close the reconstructed images are to the originals, while KL divergence ensures the latent space matches the prior.

🔧 Debug

expert

Why does this VAE training code raise a runtime error?

Consider this PyTorch training loop snippet for a VAE. Why does it raise a runtime error?

Computer Vision

for data in dataloader:
    optimizer.zero_grad()
    x = data.view(-1, 784)
    z = encoder(x)
    x_recon = decoder(z)
    recon_loss = nn.functional.binary_cross_entropy(x_recon, x)
    kl_loss = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
    loss = recon_loss + kl_loss
    loss.backward()
    optimizer.step()

A The decoder output shape does not match the input shape, causing a shape mismatch error.

B The input x is not normalized, causing the binary cross entropy to fail.

C The optimizer is not zeroed before backward pass, causing gradient accumulation errors.

D Variables mu and logvar are not defined in the training loop, causing a NameError.

💡 Hint

Check where mu and logvar come from in the encoder.

The variables mu and logvar are outputs of the encoder but are not returned or defined in this snippet, so referencing them causes an error.

Practice

1. What is the main purpose of a Variational Autoencoder (VAE) in computer vision?

easy

A. To compress data into a smaller, meaningful representation and generate new data samples

B. To classify images into different categories

C. To detect edges in images

D. To increase the resolution of images

Solution

1. Step 1: Understand VAE's role in data compression

   VAEs encode input data into a smaller latent space that captures important features.

2. Step 2: Recognize VAE's generative ability

   VAEs can generate new data by sampling from the latent space, unlike simple compression methods.

3. Final Answer:

   To compress data into a smaller, meaningful representation and generate new data samples -> Option A

4. Quick Check:

   VAE purpose = compress + generate [OK]

Hint: Remember: VAE compresses and creates new data [OK]

Common Mistakes:

• Confusing VAE with classifiers
• Thinking VAE only compresses without generation
• Mixing VAE with image enhancement methods

2. Which part of the Variational Autoencoder architecture introduces randomness during training?

easy

A. The encoder output layer

B. The reparameterization trick

C. The decoder input layer

D. The loss function calculation

Solution

1. Step 1: Identify where randomness is needed

   Randomness is introduced to sample from the latent space during training.

2. Step 2: Recognize the reparameterization trick's role

   This trick allows gradients to flow through random sampling by expressing sampling as a deterministic function plus noise.

3. Final Answer:

   The reparameterization trick -> Option B

4. Quick Check:

   Randomness in VAE = reparameterization trick [OK]

Hint: Randomness comes from reparameterization trick [OK]

Common Mistakes:

• Thinking encoder output is random by itself
• Confusing decoder input with randomness source
• Assuming loss function adds randomness

3. Given the following PyTorch snippet for sampling latent variables in a VAE, what is the shape of z if mu and log_var have shape (batch_size, latent_dim)?

std = torch.exp(0.5 * log_var)
eps = torch.randn_like(std)
z = mu + eps * std

medium

A. (latent_dim, batch_size)

B. (1, latent_dim)

C. (batch_size, 1)

D. (batch_size, latent_dim)

Solution

1. Step 1: Analyze shapes of mu and log_var

   Both have shape (batch_size, latent_dim), so std and eps will match this shape.

2. Step 2: Calculate z shape from element-wise operations

   Since z = mu + eps * std is element-wise, z keeps the same shape as mu and std.

3. Final Answer:

   (batch_size, latent_dim) -> Option D

4. Quick Check:

   z shape = mu shape = (batch_size, latent_dim) [OK]

Hint: Element-wise ops keep shape same as inputs [OK]

Common Mistakes:

• Swapping batch_size and latent_dim dimensions
• Assuming z is a vector, not a batch
• Confusing shape of eps with mu

4. You implemented a VAE but notice the generated images are very blurry. Which debugging step is most likely to improve image sharpness?

medium

A. Use a smaller batch size during training

B. Remove the KL divergence term from the loss function

C. Increase the latent space dimension to capture more details

D. Replace the decoder with a simple linear layer

Solution

1. Step 1: Understand cause of blurry images

   Blurriness often means the latent space is too small to capture enough detail.

2. Step 2: Choose the fix that increases detail capacity

   Increasing latent dimension allows encoding more features, improving sharpness.

3. Final Answer:

   Increase the latent space dimension to capture more details -> Option C

4. Quick Check:

   More latent dims = better detail [OK]

Hint: Bigger latent space means clearer images [OK]

Common Mistakes:

• Removing KL divergence breaks training stability
• Changing batch size rarely fixes blurriness
• Simplifying decoder reduces model power

5. You want to generate new images similar to your training set using a trained VAE. Which step correctly describes how to generate these images?

hard

A. Sample random points from a standard normal distribution, then pass them through the decoder

B. Feed random noise directly into the encoder and use its output as images

C. Use the encoder to compress training images, then output those compressed vectors as images

D. Train the VAE again with random noise as input to generate images

Solution

1. Step 1: Understand VAE latent space distribution

   The latent space is modeled as a standard normal distribution after training.

2. Step 2: Generate images by sampling and decoding

   Sampling random points from this distribution and decoding them produces new images similar to training data.

3. Final Answer:

   Sample random points from a standard normal distribution, then pass them through the decoder -> Option A

4. Quick Check:

   Generate = sample latent + decode [OK]

Hint: Sample latent space, decode to generate images [OK]

Common Mistakes:

• Feeding noise into encoder instead of decoder
• Using encoder outputs as images directly
• Retraining VAE unnecessarily for generation