Model optimization (quantization, pruning) in PyTorch - Model Pipeline Trace

Model Pipeline - Model optimization (quantization, pruning)

This pipeline shows how a neural network model is made smaller and faster using two techniques: quantization and pruning. Quantization reduces the number size used in the model, and pruning removes unnecessary parts of the model. This helps the model run efficiently on devices with less power.

Data Flow - 6 Stages

1Original Data Input

1000 rows x 28 x 28 pixels→Load grayscale images of handwritten digits→1000 rows x 28 x 28 pixels

Image of digit '7' represented as 28x28 pixel values

↓

2Preprocessing

1000 rows x 28 x 28 pixels→Normalize pixel values to range 0-1→1000 rows x 28 x 28 pixels

Pixel value 150 becomes 0.59

↓

3Feature Engineering

1000 rows x 28 x 28 pixels→Flatten images into 784-length vectors→1000 rows x 784 columns

28x28 image becomes a list of 784 numbers

↓

4Model Training

1000 rows x 784 columns→Train a simple neural network with 1 hidden layer→Model with weights and biases

Weights matrix shape: 784 x 128

↓

5Model Pruning

Model with weights and biases→Remove 30% of smallest weights (set to zero)→Model with 30% fewer active weights

Weights matrix now has 70% non-zero values

↓

6Model Quantization

Model with pruned weights (float32)→Convert weights from 32-bit floats to 8-bit integers→Model with quantized weights (int8)

Weight 0.1234 float32 becomes 12 int8

Training Trace - Epoch by Epoch

Loss
0.7 |*       
0.6 | *      
0.5 |  *     
0.4 |   *    
0.3 |    **  
0.2 |      * 
    +--------
     1 2 3 4 5 Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.75	Model starts learning, loss is high, accuracy moderate
2	0.45	0.85	Loss decreases, accuracy improves
3	0.35	0.90	Model converges well, good accuracy
4	0.30	0.92	Slight improvement, model stabilizes
5	0.28	0.93	Final epoch, model ready for optimization

Prediction Trace - 5 Layers

Layer 1: Input Layer

Layer 2: Hidden Layer (Dense)

Layer 3: Pruning Applied

Layer 4: Quantization Applied

Layer 5: Output Layer (Softmax)

Model Quiz - 3 Questions

What does pruning do to the model weights?

A Increases the size of weights

B Removes small weights by setting them to zero

C Changes weights to floating point numbers

D Adds more weights to the model

Key Insight

Model optimization techniques like pruning and quantization help make models smaller and faster without losing much accuracy. Pruning removes unimportant weights, and quantization reduces number size. This is important for running models on devices with limited resources.