Batch size and epochs in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Effect of Batch Size on Training Speed

Which statement best describes how increasing the batch size affects the training speed of a neural network?

A Increasing batch size always slows down training because more data is processed at once.

B Increasing batch size generally speeds up training per epoch but may require more memory.

C Batch size does not affect training speed; it only changes model accuracy.

D Smaller batch sizes always use less memory and train faster than larger batch sizes.

💡 Hint

Think about how processing more samples at once affects computation and memory.

Larger batch sizes process more samples in parallel, which can speed up training per epoch but require more memory. Smaller batches use less memory but may be slower overall.

❓ Predict Output

intermediate

Number of Weight Updates with Different Batch Sizes

Consider a dataset with 1000 samples. You train a model for 5 epochs with batch size 100. How many weight updates occur during training?

TensorFlow

dataset_size = 1000
batch_size = 100
epochs = 5
updates = (dataset_size // batch_size) * epochs
print(updates)

A 50

B 5

C 500

D 100

💡 Hint

Calculate how many batches fit in one epoch, then multiply by epochs.

Each epoch has 1000/100 = 10 batches. Over 5 epochs, total updates = 10 * 5 = 50.

❓ Hyperparameter

advanced

Choosing Batch Size for Memory Constraints

You want to train a deep neural network on a GPU with limited memory. Which batch size choice is best to avoid out-of-memory errors while maintaining reasonable training speed?

A Batch size does not affect memory usage; focus on learning rate instead.

B Use a batch size of 1 to minimize memory usage but expect slower training.

C Use the largest batch size that fits in memory without causing errors.

D Use a very large batch size to reduce the number of updates.

💡 Hint

Think about balancing memory limits and training efficiency.

The largest batch size that fits in memory maximizes GPU usage and speeds training without causing errors. Too small batches slow training; too large cause memory errors.

❓ Metrics

advanced

Effect of Epochs on Model Performance Metrics

You train a model for 10 epochs and observe training accuracy improves but validation accuracy plateaus after 5 epochs. What does this indicate?

A The model is overfitting; training longer may reduce validation accuracy.

B The model is underfitting and needs more epochs.

C The batch size is too large, causing poor validation accuracy.

D The learning rate is too high, causing unstable training.

💡 Hint

Consider what happens when training accuracy improves but validation does not.

When training accuracy improves but validation accuracy stops improving, the model is likely overfitting the training data.

🔧 Debug

expert

Identifying Epochs and Batch Size Impact in TensorFlow Training Loop

Given the code below, what will be the printed output for the number of batches processed per epoch?

TensorFlow

import tensorflow as tf

# Dataset with 120 samples
x = tf.random.normal([120, 10])
y = tf.random.uniform([120], maxval=2, dtype=tf.int32)

batch_size = 25
epochs = 3

dataset = tf.data.Dataset.from_tensor_slices((x, y)).batch(batch_size)

for epoch in range(epochs):
    batch_count = 0
    for batch_x, batch_y in dataset:
        batch_count += 1
    print(f"Epoch {epoch+1} batches: {batch_count}")

A

Epoch 1 batches: 6
Epoch 2 batches: 6
Epoch 3 batches: 6

B

Epoch 1 batches: 4
Epoch 2 batches: 4
Epoch 3 batches: 4

C

Epoch 1 batches: 3
Epoch 2 batches: 3
Epoch 3 batches: 3

D

Epoch 1 batches: 5
Epoch 2 batches: 5
Epoch 3 batches: 5

💡 Hint

Calculate how many batches of size 25 fit into 120 samples.

120 samples divided by batch size 25 gives 4 full batches (25*4=100) plus 1 batch with remaining 20 samples, totaling 5 batches per epoch.

Practice

1. What does the batch size control during training in TensorFlow?

easy

A. The total number of times the model sees the entire dataset

B. The number of samples processed before the model updates its weights

C. The number of layers in the neural network

D. The learning rate of the optimizer

Solution

1. Step 1: Understand batch size meaning

   Batch size is how many samples the model processes before updating weights.

2. Step 2: Differentiate from epochs

   Epochs count full dataset passes, not batch updates.

3. Final Answer:

   The number of samples processed before the model updates its weights -> Option B

4. Quick Check:

   Batch size = samples per update [OK]

Hint: Batch size = samples per update, epochs = full dataset passes [OK]

Common Mistakes:

• Confusing batch size with epochs
• Thinking batch size controls learning rate
• Mixing batch size with model layers

2. Which of the following is the correct way to set batch size and epochs in TensorFlow's model.fit() method?

easy

A. model.fit(x_train, y_train, batch_size=32, epochs=10)

B. model.fit(x_train, y_train, batch=32, epochs=10)

C. model.fit(x_train, y_train, batchsize=32, epoch=10)

D. model.fit(x_train, y_train, size_batch=32, epochs=10)

Solution

1. Step 1: Recall correct parameter names

   TensorFlow uses batch_size and epochs as parameter names in model.fit().

2. Step 2: Check each option

   Only model.fit(x_train, y_train, batch_size=32, epochs=10) uses correct parameter names exactly.

3. Final Answer:

   model.fit(x_train, y_train, batch_size=32, epochs=10) -> Option A

4. Quick Check:

   Correct parameter names = batch_size, epochs [OK]

Hint: Use exact parameter names: batch_size and epochs [OK]

Common Mistakes:

• Using batch instead of batch_size
• Using epoch instead of epochs
• Misspelling parameter names

3. Consider this code snippet:

history = model.fit(x_train, y_train, batch_size=64, epochs=3, verbose=0)
print(len(history.history['loss']))

What will be the printed output?

medium

A. 64

B. 1

C. 3

D. Number of samples in x_train

Solution

1. Step 1: Understand what history.history['loss'] stores

   It stores loss values per epoch, so its length equals number of epochs.

2. Step 2: Check epochs parameter

   Epochs is set to 3, so length will be 3.

3. Final Answer:

   3 -> Option C

4. Quick Check:

   Length of loss history = epochs = 3 [OK]

Hint: Loss history length equals epochs count [OK]

Common Mistakes:

• Confusing batch size with number of loss entries
• Thinking loss history length equals dataset size
• Assuming one loss per batch instead of per epoch

4. You wrote this code but it runs very slowly:

model.fit(x_train, y_train, batch_size=1, epochs=10)

What is the most likely reason for the slow training?

medium

A. Using batch_size=1 disables GPU acceleration

B. Epochs set to 10 is too low to train well

C. Batch size should be larger than number of epochs

D. Batch size of 1 causes frequent weight updates, slowing training

Solution

1. Step 1: Understand effect of batch size 1

   Batch size 1 means model updates weights after every single sample, causing overhead.

2. Step 2: Evaluate other options

   Epochs=10 is normal; batch size does not need to be larger than epochs; batch size 1 does not disable GPU.

3. Final Answer:

   Batch size of 1 causes frequent weight updates, slowing training -> Option D

4. Quick Check:

   Small batch size = slower training due to many updates [OK]

Hint: Very small batch size slows training due to many updates [OK]

Common Mistakes:

• Thinking epochs number causes slowness
• Believing batch size must be bigger than epochs
• Assuming batch size disables GPU

5. You have a dataset of 10,000 samples. You want to train a model efficiently and avoid overfitting. Which combination of batch size and epochs is best?

hard

A. Batch size = 1000, epochs = 5

B. Batch size = 10, epochs = 1000

C. Batch size = 1, epochs = 10000

D. Batch size = 500, epochs = 50

Solution

1. Step 1: Consider batch size impact

   Large batch sizes (like 1000) speed training and provide stable updates.

2. Step 2: Consider epochs and overfitting

   Too many epochs (like 1000 or 10000) risk overfitting; fewer epochs with larger batches balance training.

3. Step 3: Evaluate options

   Batch size = 1000, epochs = 5 balances batch size and epochs for efficient training and less overfitting.

4. Final Answer:

   Batch size = 1000, epochs = 5 -> Option A

5. Quick Check:

   Balanced batch size and epochs avoid overfitting [OK]

Hint: Large batch + fewer epochs = efficient, less overfitting [OK]

Common Mistakes:

• Choosing very small batch sizes with many epochs
• Ignoring overfitting risk with too many epochs
• Assuming bigger batch size always means better accuracy