Loss functions (MSE, cross-entropy) in TensorFlow - Model Pipeline Trace

Model Pipeline - Loss functions (MSE, cross-entropy)

This pipeline shows how a simple neural network learns to predict numbers using two common loss functions: Mean Squared Error (MSE) for regression and Cross-Entropy for classification. The loss functions measure how far the model's guesses are from the true answers, helping the model improve step by step.

Data Flow - 5 Stages

1Data Input

1000 rows x 10 columns→Raw data loaded with 10 features per example→1000 rows x 10 columns

[[0.5, 1.2, ..., 0.3], [0.1, 0.4, ..., 0.9], ...]

↓

2Preprocessing

1000 rows x 10 columns→Normalize features to range 0-1→1000 rows x 10 columns

[[0.05, 0.12, ..., 0.03], [0.01, 0.04, ..., 0.09], ...]

↓

3Train/Test Split

1000 rows x 10 columns→Split data into 800 training and 200 testing rows→Train: 800 rows x 10 columns, Test: 200 rows x 10 columns

Train features shape: (800, 10), Test features shape: (200, 10)

↓

4Model Training

800 rows x 10 columns→Feed data into neural network, compute loss (MSE or Cross-Entropy), update weights→Model weights updated, loss value computed per epoch

Epoch 1 loss: 0.5, Epoch 10 loss: 0.1

↓

5Prediction

1 row x 10 columns→Model predicts output for new input→1 row x 1 column (regression) or 1 row x 3 columns (classification probabilities)

[0.75] for regression or [0.1, 0.7, 0.2] for classification

Training Trace - Epoch by Epoch

Loss
0.5 |***************
0.4 |**********
0.3 |*******
0.2 |****
0.1 |**
0.0 +----------------
     1  3  5  7  10 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.48	0.60	Loss starts high; accuracy is low as model begins learning
3	0.30	0.75	Loss decreases; accuracy improves as model adjusts weights
5	0.18	0.85	Loss continues to drop; model predictions become more accurate
7	0.12	0.90	Loss decreases steadily; accuracy nearing good performance
10	0.08	0.93	Loss low; accuracy high, model well trained

Prediction Trace - 4 Layers

Layer 1: Input Layer

Layer 2: Hidden Layer (ReLU activation)

Layer 3: Output Layer (Regression - no activation)

Layer 4: Loss Calculation (MSE)

Model Quiz - 3 Questions

What does the Mean Squared Error (MSE) loss measure?

A The sum of all prediction values

B The probability that the prediction is correct

C The average squared difference between predicted and true values

D The difference between input and output shapes

Key Insight

Loss functions like MSE and Cross-Entropy guide the model to learn by showing how far its predictions are from the truth. Watching loss decrease and accuracy increase over time tells us the model is improving.