Neural network architecture in ML Python - Model Pipeline Trace

Model Pipeline - Neural network architecture

This pipeline shows how a simple neural network learns to recognize handwritten digits. It starts with raw images, processes them through layers of neurons, and improves its guesses over time.

Data Flow - 4 Stages

1Input Layer

1000 rows x 28 x 28 pixels→Raw grayscale images of handwritten digits→1000 rows x 784 features

Image of digit '7' flattened into a 784-length vector

↓

2Hidden Layer 1

1000 rows x 784 features→Fully connected layer with 128 neurons and ReLU activation→1000 rows x 128 features

Vector of 128 positive values representing learned features

↓

3Hidden Layer 2

1000 rows x 128 features→Fully connected layer with 64 neurons and ReLU activation→1000 rows x 64 features

Vector of 64 positive values representing deeper features

↓

4Output Layer

1000 rows x 64 features→Fully connected layer with 10 neurons and softmax activation→1000 rows x 10 classes

Probabilities for digits 0 to 9, e.g., [0.01, 0.02, ..., 0.85, ...]

Training Trace - Epoch by Epoch

Loss
1.2 |*       
1.0 | *      
0.8 |  *     
0.6 |   *    
0.4 |    *   
0.2 |     *  
0.0 +---------
      1 2 3 4 5
       Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.55	Model starts learning with moderate accuracy
2	0.8	0.72	Loss decreases and accuracy improves
3	0.6	0.80	Model is learning important features
4	0.45	0.85	Accuracy continues to increase steadily
5	0.35	0.89	Model converges with good accuracy

Prediction Trace - 4 Layers

Layer 1: Input Layer

Layer 2: Hidden Layer 1 (ReLU)

Layer 3: Hidden Layer 2 (ReLU)

Layer 4: Output Layer (Softmax)

Model Quiz - 3 Questions

What does the ReLU activation function do in the hidden layers?

A It changes all negative values to zero

B It normalizes values between 0 and 1

C It outputs probabilities that sum to 1

D It flips negative values to positive

Key Insight

A neural network learns by adjusting weights in layers to reduce loss and improve accuracy. Activation functions like ReLU help the model learn complex patterns, while softmax converts outputs into understandable probabilities.