Batch normalization in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why use batch normalization in training?

Batch normalization helps improve training by:

• Avoiding vanishing gradients
• Reducing internal covariate shift
• Increasing model capacity
• Replacing dropout

Which is the main reason batch normalization is used?

A It reduces internal covariate shift by normalizing layer inputs during training.

B It increases the number of neurons in each layer automatically.

C It replaces the need for activation functions like ReLU.

D It prevents overfitting by randomly dropping neurons.

💡 Hint

Think about what happens to the input distribution of layers during training.

Batch normalization normalizes the inputs to each layer to keep their distribution stable, which helps training converge faster and more reliably.

❓ Predict Output

intermediate

Output shape after batch normalization layer

Given the following TensorFlow code, what is the shape of output?

TensorFlow

import tensorflow as tf
input_tensor = tf.random.normal([32, 28, 28, 3])
bn_layer = tf.keras.layers.BatchNormalization()
output = bn_layer(input_tensor)
output_shape = output.shape

A (32, 28, 28, 3)

B (28, 28, 3)

C (32, 3)

D (32, 28, 28)

💡 Hint

Batch normalization does not change the shape of the input tensor.

The batch normalization layer normalizes each feature but keeps the input shape unchanged.

❓ Hyperparameter

advanced

Choosing momentum value in batch normalization

In TensorFlow's BatchNormalization layer, the momentum parameter controls the moving average of mean and variance. Which statement about setting momentum is true?

A A higher momentum means the moving averages update quickly, relying more on current batch values.

B Momentum controls the learning rate of the batch normalization layer.

C A higher momentum means the moving averages update slowly, relying more on past values.

D Momentum is only used during training and ignored during inference.

💡 Hint

Think about how moving averages work with momentum values close to 1.

Momentum near 1 means the moving averages change slowly, keeping more of the past information. Lower momentum updates faster with new batch statistics.

🔧 Debug

advanced

Why does batch normalization cause error with batch size 1?

Consider this code snippet:

import tensorflow as tf
input_tensor = tf.random.normal([1, 10])
bn = tf.keras.layers.BatchNormalization()
output = bn(input_tensor)

Running this raises an error. Why?

A Batch normalization requires input values to be positive.

B Batch normalization requires batch size > 1 to compute variance; batch size 1 causes division by zero.

C Batch normalization only works with 4D tensors, not 2D tensors.

D Batch normalization cannot be used on tensors created with tf.random.normal.

💡 Hint

Think about how variance is calculated with only one sample.

With batch size 1, variance calculation is not possible (division by zero), causing batch normalization to fail.

❓ Metrics

expert

Effect of batch normalization on training metrics

You train two identical neural networks on the same data. One uses batch normalization after each dense layer; the other does not. Which of the following is the most likely difference in training metrics?

A Batch normalization causes the training loss to increase and accuracy to decrease.

B The network without batch normalization converges faster but has lower validation accuracy.

C Both networks converge at the same speed but the one without batch normalization has better accuracy.

D The network with batch normalization converges faster and achieves higher accuracy on training and validation sets.

💡 Hint

Batch normalization stabilizes and speeds up training.

Batch normalization reduces internal covariate shift, allowing faster convergence and often better accuracy.