Batching and shuffling in TensorFlow - Model Pipeline Trace

Model Pipeline - Batching and shuffling

This pipeline shows how data is prepared by grouping it into batches and mixing the order randomly before training a model. This helps the model learn better by seeing varied examples in each step.

Data Flow - 3 Stages

1Raw dataset

1000 rows x 10 columns→Initial dataset with 1000 samples and 10 features each→1000 rows x 10 columns

[[0.5, 1.2, ..., 0.3], [0.7, 0.8, ..., 0.1], ...]

↓

2Shuffle dataset

1000 rows x 10 columns→Randomly reorder all 1000 samples→1000 rows x 10 columns

[[0.7, 0.8, ..., 0.1], [0.5, 1.2, ..., 0.3], ...]

↓

3Batch dataset

1000 rows x 10 columns→Group samples into batches of 100→10 batches x 100 rows x 10 columns

[Batch 1: [[0.7, 0.8, ..., 0.1], ... 100 samples], Batch 2: [...]]

Training Trace - Epoch by Epoch

Loss
1.0 |*         
0.8 | **       
0.6 |  ***     
0.4 |    ****  
0.2 |      *** 
0.0 +---------
      1 2 3 4 5
      Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.85	0.60	Loss starts high; accuracy is low as model begins learning
2	0.65	0.72	Loss decreases; accuracy improves as model sees shuffled batches
3	0.50	0.80	Model learns better with varied batches; loss drops further
4	0.40	0.85	Continued improvement; shuffling helps avoid overfitting
5	0.35	0.88	Loss stabilizes; accuracy nears good performance

Prediction Trace - 4 Layers

Layer 1: Input batch

Layer 2: Model forward pass

Layer 3: Loss calculation

Layer 4: Backpropagation and update

Model Quiz - 3 Questions

Why do we shuffle data before batching?

A To reduce the number of samples

B To mix samples so the model sees varied data each batch

C To increase the batch size

D To sort samples by label

Key Insight

Batching groups data into manageable pieces for training, and shuffling mixes data to help the model learn patterns better by avoiding seeing similar samples in a row. This leads to smoother training with improving accuracy and decreasing loss.