Autoencoders in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Model Choice

intermediate

Choosing the right autoencoder architecture

You want to compress images of size 28x28 pixels into a smaller representation and then reconstruct them. Which autoencoder architecture is best suited for this task?

A A GAN (Generative Adversarial Network) architecture

B A recurrent autoencoder using LSTM layers

C A fully connected autoencoder with only dense layers

D A convolutional autoencoder with convolutional layers in encoder and decoder

💡 Hint

Think about the type of data and which layers capture spatial features best.

Convolutional autoencoders are best for image data because convolutional layers capture spatial patterns effectively. Dense layers lose spatial information, and recurrent layers are for sequences. GANs are for generation, not compression.

❓ Predict Output

intermediate

Output shape of encoded representation

Given this TensorFlow autoencoder encoder code snippet, what is the shape of the encoded output?

TensorFlow

import tensorflow as tf
encoder = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28,28)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(32, activation='relu')
])
encoded = encoder(tf.random.uniform((10,28,28)))
print(encoded.shape)

A (10, 64)

B (32, 10)

C (10, 32)

D (10, 28, 28)

💡 Hint

Look at the last Dense layer's output units and batch size.

The input batch size is 10. The last Dense layer outputs 32 units, so the output shape is (10, 32).

❓ Hyperparameter

advanced

Choosing the right latent dimension size

You are training an autoencoder on tabular data. What is the effect of choosing a very small latent dimension size?

A The model may underfit and fail to reconstruct input data well

B The model will always overfit the training data

C The training time will be longer but reconstruction improves

D The model will ignore the input data and output random noise

💡 Hint

Think about how much information the latent space can hold.

A very small latent dimension limits the information the model can store, causing underfitting and poor reconstruction.

❓ Metrics

advanced

Evaluating autoencoder reconstruction quality

Which metric is most appropriate to measure how well an autoencoder reconstructs input images?

A Cross-entropy loss between input and output images

B Mean Squared Error (MSE) between input and output images

C Accuracy of classification on reconstructed images

D Precision and recall of reconstructed images

💡 Hint

Consider a metric that measures pixel-wise difference.

Mean Squared Error measures the average squared difference between input and reconstructed pixels, making it suitable for reconstruction quality.

🔧 Debug

expert

Identifying the cause of poor autoencoder training

You trained an autoencoder but the loss does not decrease during training. What is the most likely cause?

A The activation function in the output layer is missing or inappropriate for the data range

B The input data is normalized between 0 and 1 but the model expects raw pixel values

C The optimizer is set to 'sgd' instead of 'adam' which always fails

D The batch size is too large causing unstable gradients

💡 Hint

Check if the output layer activation matches the input data scale.

If the output layer activation is missing or not suitable (e.g., no sigmoid for data scaled 0-1), the model cannot learn proper reconstruction, causing loss to stagnate.

Practice

1. What is the main purpose of an autoencoder in machine learning?

easy

A. To compress data and then reconstruct it to learn important features

B. To classify images into categories

C. To generate new data from random noise

D. To sort data in ascending order

Solution

1. Step 1: Understand the role of autoencoders

   Autoencoders are designed to learn a compressed representation of data by encoding and then decoding it.

2. Step 2: Identify the main goal

   The goal is to capture important features by reconstructing the input data as closely as possible.

3. Final Answer:

   To compress data and then reconstruct it to learn important features -> Option A

4. Quick Check:

   Autoencoder purpose = compress and reconstruct [OK]

Hint: Autoencoders compress and rebuild data to learn features [OK]

Common Mistakes:

• Confusing autoencoders with classifiers
• Thinking autoencoders generate new random data
• Assuming autoencoders sort or label data

2. Which of the following is the correct way to define the encoder part of a simple autoencoder in TensorFlow using Keras Sequential API?

easy

A. encoder = tf.keras.Sequential([tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(784,))])

B. encoder = tf.keras.Sequential([tf.keras.layers.Dense(32, activation='softmax', input_shape=(784,))])

C. encoder = tf.keras.Sequential([tf.keras.layers.Dense(32, activation='relu', input_shape=(784,))])

D. encoder = tf.keras.Sequential([tf.keras.layers.Dense(32)])

Solution

1. Step 1: Check the layer type and activation

   The encoder should use a Dense layer with 'relu' activation for feature extraction.

2. Step 2: Verify input shape and activation correctness

   Input shape (784,) matches flattened input, and 'relu' is suitable for encoding.

3. Final Answer:

   encoder = tf.keras.Sequential([tf.keras.layers.Dense(32, activation='relu', input_shape=(784,))]) -> Option C

4. Quick Check:

   Dense + relu + correct input shape = encoder = tf.keras.Sequential([tf.keras.layers.Dense(32, activation='relu', input_shape=(784,))]) [OK]

Hint: Use Dense with relu and correct input shape for encoder [OK]

Common Mistakes:

• Using softmax activation in encoder
• Using Conv2D layer with flat input shape
• Omitting activation function

3. Given the following TensorFlow code for an autoencoder, what will be the shape of the encoded output?

import tensorflow as tf
input_layer = tf.keras.layers.Input(shape=(28, 28, 1))
flat = tf.keras.layers.Flatten()(input_layer)
encoded = tf.keras.layers.Dense(16, activation='relu')(flat)
model = tf.keras.Model(inputs=input_layer, outputs=encoded)

sample_input = tf.random.normal([10, 28, 28, 1])
encoded_output = model(sample_input)
print(encoded_output.shape)

medium

A. (16, 10)

B. (10, 28, 28, 1)

C. (10, 784)

D. (10, 16)

Solution

1. Step 1: Analyze input and flatten layer

   Input shape is (28, 28, 1), flatten converts it to (784,) per sample.

2. Step 2: Check Dense layer output size

   The Dense layer outputs 16 units, so output shape per sample is (16,).

3. Step 3: Consider batch size

   Batch size is 10, so final output shape is (10, 16).

4. Final Answer:

   (10, 16) -> Option D

5. Quick Check:

   Batch 10 x Dense 16 units = (10, 16) [OK]

Hint: Flatten input then Dense units define output shape [OK]

Common Mistakes:

• Confusing input shape with output shape
• Ignoring batch size dimension
• Assuming output shape matches input shape

4. You wrote this autoencoder training code but get a shape mismatch error:

autoencoder.compile(optimizer='adam', loss='mse')
autoencoder.fit(x_train, x_train.reshape(-1, 784), epochs=10)

What is the likely cause?

medium

A. Input and output shapes do not match during training

B. Loss function 'mse' is not suitable for autoencoders

C. Optimizer 'adam' cannot be used with autoencoders

D. Epochs value 10 is too low

Solution

1. Step 1: Check input and output shapes in fit()

   Input is x_train, output is x_train reshaped to (-1, 784), shapes differ if model expects original shape.

2. Step 2: Understand autoencoder training requirement

   Autoencoder input and output shapes must match; reshaping output differently causes mismatch.

3. Final Answer:

   Input and output shapes do not match during training -> Option A

4. Quick Check:

   Input/output shape mismatch causes error [OK]

Hint: Ensure input and output shapes match exactly during training [OK]

Common Mistakes:

• Changing output shape differently from input
• Blaming optimizer or loss function incorrectly
• Ignoring shape requirements in fit()

5. You want to build an autoencoder to denoise images. Which approach best fits this task?

hard

A. Train the autoencoder with clean images as input and noisy images as output

B. Train the autoencoder with noisy images as input and clean images as output

C. Train the autoencoder only on clean images without noise

D. Train the autoencoder with random noise as both input and output

Solution

1. Step 1: Understand denoising autoencoder goal

   Denoising autoencoders learn to remove noise by mapping noisy input to clean output.

2. Step 2: Choose training data setup

   Input should be noisy images, output should be corresponding clean images for learning noise removal.

3. Final Answer:

   Train the autoencoder with noisy images as input and clean images as output -> Option B

4. Quick Check:

   Denoising = noisy input, clean output [OK]

Hint: Noisy input, clean output trains denoising autoencoder [OK]

Common Mistakes:

• Swapping input and output roles
• Training only on clean images without noise
• Using random noise as output