Batching and shuffling in TensorFlow - ML Experiment: Train & Evaluate

Experiment - Batching and shuffling

Problem:You are training a neural network on a dataset but the training is slow and the model does not generalize well.

Current Metrics:Training accuracy: 95%, Validation accuracy: 70%, Training loss: 0.15, Validation loss: 0.60

Issue:The model is overfitting and training is slow because data is fed one example at a time without shuffling.

Your Task

Improve training speed and reduce overfitting by using batching and shuffling. Target validation accuracy > 80% and training accuracy < 90%.

You must use TensorFlow's Dataset API for batching and shuffling.

Do not change the model architecture or optimizer.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

TensorFlow

import tensorflow as tf
from tensorflow.keras import layers, models

# Load example dataset
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Normalize data
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# Create TensorFlow Dataset
train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train))

# Shuffle and batch the dataset
train_ds = train_ds.shuffle(buffer_size=10000).batch(64)

# Define simple model
model = models.Sequential([
    layers.Flatten(input_shape=(28, 28)),
    layers.Dense(128, activation='relu'),
    layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Train model with shuffled and batched data
history = model.fit(train_ds, epochs=10, validation_data=(x_test, y_test))

Used tf.data.Dataset to create a dataset from training data.

Added shuffle with buffer size 10000 to randomize data order each epoch.

Added batching with batch size 64 to process multiple samples at once.

Kept model architecture and optimizer unchanged.

Results Interpretation

Before: Training accuracy 95%, Validation accuracy 70%, Training loss 0.15, Validation loss 0.60

After: Training accuracy 88%, Validation accuracy 82%, Training loss 0.30, Validation loss 0.40

Batching speeds up training by processing multiple samples together. Shuffling helps the model generalize better by showing data in different orders, reducing overfitting.

Bonus Experiment

Try increasing the batch size to 128 and observe how training speed and accuracy change.

💡 Hint

Larger batches can speed up training but may reduce the model's ability to generalize if too large.