Model optimization (quantization, pruning) in PyTorch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is model quantization in machine learning?

Model quantization is a technique that reduces the precision of the numbers used to represent a model's parameters, typically from 32-bit floats to lower-bit integers, to make the model smaller and faster without much loss in accuracy.

beginner

Explain model pruning in simple terms.

Model pruning means removing parts of a neural network, like some weights or neurons, that are less important. This makes the model smaller and faster while trying to keep its performance close to the original.

intermediate

How does quantization help in deploying models on mobile devices?

Quantization reduces the model size and speeds up computation by using fewer bits for numbers. This saves memory and battery on mobile devices, making models run faster and use less power.

intermediate

What is the main trade-off when applying pruning to a neural network?

The main trade-off is between model size and accuracy. Pruning reduces size and speeds up the model but can cause some loss in accuracy if important parts are removed.

intermediate

Name two common types of pruning methods.

Two common pruning methods are: 1) Magnitude-based pruning, which removes weights with small values, and 2) Structured pruning, which removes entire neurons or filters.

What does quantization primarily reduce in a neural network model?

A The number of training samples

B The number of layers

C The learning rate

D The precision of numbers used for weights

Quantization reduces the precision of the numbers representing weights, typically from 32-bit floats to lower-bit integers.

Which of the following is a goal of pruning a model?

A Increase model size

B Remove less important weights or neurons

C Add more layers

D Increase training time

Pruning removes less important weights or neurons to make the model smaller and faster.

What is a common effect of quantization on model accuracy?

A It may slightly reduce accuracy

B It has no effect on accuracy

C It always improves accuracy

D It causes the model to forget training data

Quantization can slightly reduce accuracy due to lower precision but often the loss is minimal.

Which pruning method removes entire neurons or filters?

A Structured pruning

B Magnitude-based pruning

C Random pruning

D Gradient pruning

Structured pruning removes entire neurons or filters, unlike magnitude-based pruning which removes individual weights.

Why is model optimization important for edge devices?

A To increase model complexity

B To use more power

C To reduce memory and computation needs

D To make training slower

Optimization reduces memory and computation, making models suitable for devices with limited resources.

Describe in your own words what model quantization and pruning are, and why they are useful.

Think about how to make a big model smaller and faster without losing much accuracy.

Explain the trade-offs involved when applying pruning to a neural network model.

Consider what happens if you remove too much from the model.