Model optimization (quantization, pruning) in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is model quantization in machine learning?

Model quantization is a technique that reduces the precision of the numbers used to represent a model's parameters, typically from 32-bit floats to lower-bit integers, to make the model smaller and faster without much loss in accuracy.

beginner

Explain model pruning in simple terms.

Model pruning means removing parts of a neural network, like some weights or neurons, that are less important. This makes the model smaller and faster while trying to keep its performance close to the original.

intermediate

How does quantization help in deploying models on mobile devices?

Quantization reduces the model size and speeds up computation by using fewer bits for numbers. This saves memory and battery on mobile devices, making models run faster and use less power.

intermediate

What is the main trade-off when applying pruning to a neural network?

The main trade-off is between model size and accuracy. Pruning reduces size and speeds up the model but can cause some loss in accuracy if important parts are removed.

intermediate

Name two common types of pruning methods.

Two common pruning methods are: 1) Magnitude-based pruning, which removes weights with small values, and 2) Structured pruning, which removes entire neurons or filters.

What does quantization primarily reduce in a neural network model?

A The number of training samples

B The number of layers

C The learning rate

D The precision of numbers used for weights

Quantization reduces the precision of the numbers representing weights, typically from 32-bit floats to lower-bit integers.

Which of the following is a goal of pruning a model?

A Increase model size

B Remove less important weights or neurons

C Add more layers

D Increase training time

Pruning removes less important weights or neurons to make the model smaller and faster.

What is a common effect of quantization on model accuracy?

A It may slightly reduce accuracy

B It has no effect on accuracy

C It always improves accuracy

D It causes the model to forget training data

Quantization can slightly reduce accuracy due to lower precision but often the loss is minimal.

Which pruning method removes entire neurons or filters?

A Structured pruning

B Magnitude-based pruning

C Random pruning

D Gradient pruning

Structured pruning removes entire neurons or filters, unlike magnitude-based pruning which removes individual weights.

Why is model optimization important for edge devices?

A To increase model complexity

B To use more power

C To reduce memory and computation needs

D To make training slower

Optimization reduces memory and computation, making models suitable for devices with limited resources.

Describe in your own words what model quantization and pruning are, and why they are useful.

Think about how to make a big model smaller and faster without losing much accuracy.

Explain the trade-offs involved when applying pruning to a neural network model.

Consider what happens if you remove too much from the model.

Practice

1. What is the main goal of model pruning in PyTorch?

easy

A. Add more training data to improve accuracy

B. Increase the number of layers in the model

C. Change the model architecture to a different type

D. Remove less important weights to make the model smaller

Solution

1. Step 1: Understand pruning concept

   Pruning means removing weights that do not contribute much to the model's output.

2. Step 2: Identify pruning effect

   Removing these weights makes the model smaller and faster without changing its structure.

3. Final Answer:

   Remove less important weights to make the model smaller -> Option D

4. Quick Check:

   Pruning = Remove less important weights [OK]

Hint: Pruning cuts unimportant weights to shrink model size [OK]

Common Mistakes:

• Thinking pruning adds layers
• Confusing pruning with data augmentation
• Believing pruning changes model type

2. Which PyTorch function is used to apply pruning to a layer?

easy

A. torch.nn.utils.prune.l1_unstructured

B. torch.quantization.quantize_dynamic

C. torch.optim.Adam

D. torch.nn.functional.relu

Solution

1. Step 1: Identify pruning functions in PyTorch

   PyTorch pruning utilities are in torch.nn.utils.prune module.

2. Step 2: Match pruning function

   l1_unstructured is a common pruning method that removes weights based on L1 norm.

3. Final Answer:

   torch.nn.utils.prune.l1_unstructured -> Option A

4. Quick Check:

   Pruning function = l1_unstructured [OK]

Hint: Pruning uses torch.nn.utils.prune functions [OK]

Common Mistakes:

• Confusing pruning with quantization functions
• Choosing optimizer functions by mistake
• Selecting activation functions instead of pruning

3. What will be the output shape of the model's weight tensor after applying this pruning code?

import torch
import torch.nn.utils.prune as prune

layer = torch.nn.Linear(4, 3)
prune.l1_unstructured(layer, name='weight', amount=0.5)
print(layer.weight.shape)

medium

A. torch.Size([4, 3]) with half weights removed

B. torch.Size([3, 4]) with half weights removed physically

C. torch.Size([3, 4]) with some weights zeroed

D. torch.Size([4, 3]) unchanged

Solution

1. Step 1: Check layer weight shape

   A Linear layer with input 4 and output 3 has weight shape (3, 4).

2. Step 2: Understand pruning effect on weight tensor

   Pruning sets some weights to zero but does not change tensor shape.

3. Final Answer:

   torch.Size([3, 4]) with some weights zeroed -> Option C

4. Quick Check:

   Pruning zeros weights, shape stays same [OK]

Hint: Pruning zeros weights but keeps tensor shape [OK]

Common Mistakes:

• Thinking pruning changes tensor shape
• Confusing input/output dimensions order
• Assuming weights are physically removed

4. Identify the error in this PyTorch pruning code:

import torch.nn.utils.prune as prune
layer = torch.nn.Linear(5, 2)
prune.l1_unstructured(layer, name='bias', amount=0.3)

medium

A. Missing import for torch

B. Layer input and output sizes are invalid

C. Amount must be an integer, not float

D. Cannot prune bias with l1_unstructured

Solution

1. Step 1: Examine the code imports

   The code imports only prune but attempts to use torch.nn.Linear without importing torch.

2. Step 2: Identify the runtime error

   This causes NameError: name 'torch' is not defined when creating the layer.

3. Final Answer:

   Missing import for torch -> Option A

4. Quick Check:

   Missing torch import causes NameError [OK]

Hint: Import torch before using torch.nn [OK]

Common Mistakes:

• Overlooking missing torch import
• Thinking pruning bias is unsupported
• Assuming amount must be integer

5. You want to speed up inference on a PyTorch model by reducing number precision. Which approach should you use?

hard

A. Increase batch size during training

B. Apply dynamic quantization to convert weights to lower precision

C. Prune 90% of the model weights randomly

D. Add dropout layers to the model

Solution

1. Step 1: Understand quantization purpose

   Quantization reduces number precision (e.g., float32 to int8) to speed up inference.

2. Step 2: Identify correct method

   Dynamic quantization in PyTorch converts weights to lower precision without retraining.

3. Step 3: Compare other options

   Pruning removes weights but may not speed inference as much; batch size and dropout affect training, not inference speed.

4. Final Answer:

   Apply dynamic quantization to convert weights to lower precision -> Option B

5. Quick Check:

   Quantization = lower precision for speed [OK]

Hint: Quantization reduces precision to speed inference [OK]

Common Mistakes:

• Confusing pruning with quantization
• Thinking batch size affects model speed directly
• Adding dropout to speed inference