Forward pass computation in PyTorch - Model Pipeline Trace

Model Pipeline - Forward pass computation

This pipeline shows how data moves through a simple neural network during the forward pass. It transforms input data step-by-step to produce predictions.

Data Flow - 5 Stages

1Input Layer

1 row x 3 columns→Receive raw input features→1 row x 3 columns

[0.5, 1.0, -1.5]

↓

2Linear Layer 1

1 row x 3 columns→Multiply input by weights and add bias→1 row x 4 columns

Input [0.5,1.0,-1.5] * weights + bias = [0.1, 0.2, 0.3, 0.4]

↓

3ReLU Activation

1 row x 4 columns→Apply ReLU to set negative values to zero→1 row x 4 columns

ReLU([0.1, 0.2, 0.3, 0.4]) = [0.1, 0.2, 0.3, 0.4]

↓

4Linear Layer 2

1 row x 4 columns→Multiply by weights and add bias for output→1 row x 2 columns

[0.1, 0.2, 0.3, 0.4] * weights + bias = [0.5, -0.1]

↓

5Softmax Output

1 row x 2 columns→Convert raw scores to probabilities→1 row x 2 columns

Softmax([0.5, -0.1]) = [0.65, 0.35]

Training Trace - Epoch by Epoch

Loss
1.2 |*****
0.9 |****
0.7 |***
0.5 |**
0.4 |*

Epoch	Loss ↓	Accuracy ↑	Observation
1	1.2	0.45	Loss starts high, accuracy low as model begins learning
2	0.9	0.60	Loss decreases, accuracy improves
3	0.7	0.72	Model learns better features, accuracy rises
4	0.5	0.80	Loss continues to drop, accuracy nearing good performance
5	0.4	0.85	Training converges with low loss and high accuracy

Prediction Trace - 5 Layers

Layer 1: Input Layer

Layer 2: Linear Layer 1

Layer 3: ReLU Activation

Layer 4: Linear Layer 2

Layer 5: Softmax Output

Model Quiz - 3 Questions

What does the ReLU activation do during the forward pass?

A Multiplies inputs by weights

B Converts scores to probabilities

C Sets negative values to zero and keeps positive values

D Adds bias to inputs

Key Insight

The forward pass transforms input data through layers using weights, biases, and activations to produce meaningful predictions. Each step changes the data shape and values, preparing it for the next layer until final probabilities are computed.