Model optimization (quantization, pruning) in TensorFlow - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is model quantization in machine learning?

Model quantization is a technique that reduces the size of a model by converting its weights and activations from high precision (like 32-bit floats) to lower precision (like 8-bit integers). This makes the model faster and smaller, which is helpful for running on devices with limited resources.

beginner

Explain model pruning in simple terms.

Model pruning means removing parts of a neural network that are not very important, like cutting away small or unused connections. This makes the model smaller and faster without losing much accuracy.

intermediate

How does quantization affect model accuracy?

Quantization can slightly reduce model accuracy because it uses fewer bits to represent numbers, which can cause small errors. However, with careful techniques, the accuracy loss is often very small.

beginner

What is the main benefit of pruning a model?

The main benefit of pruning is to make the model smaller and faster by removing unnecessary parts, which helps when deploying models on devices with limited memory and computing power.

intermediate

Name two common types of quantization used in TensorFlow model optimization.

Two common types are post-training quantization (applied after training) and quantization-aware training (where the model learns to handle quantization during training).

What does model quantization primarily reduce?

A Model size and computation

B Training time

C Number of layers

D Dataset size

Quantization reduces model size and speeds up computation by using lower precision numbers.

Which technique removes less important connections in a neural network?

A Quantization

B Pruning

C Data augmentation

D Batch normalization

Pruning removes less important connections to make the model smaller and faster.

Which quantization method involves training the model to handle lower precision?

A Quantization-aware training

B Post-training quantization

C Pruning

D Transfer learning

Quantization-aware training teaches the model during training to work well with quantized weights.

What is a common side effect of quantization on model performance?

A Significant accuracy increase

B More memory usage

C Slight accuracy decrease

D Longer training time

Quantization can cause a slight drop in accuracy due to reduced precision.

Why is model pruning useful for mobile devices?

A It adds more layers

B It increases model complexity

C It improves data collection

D It reduces model size and speeds up inference

Pruning reduces model size and speeds up inference, which is important for devices with limited resources.

Describe in your own words what model quantization is and why it is useful.

Think about how using fewer bits for numbers can help run models on phones.

Explain how pruning helps optimize a neural network model.

Imagine trimming a tree by cutting small branches that don't help much.

Practice

1. What is the main goal of model pruning in TensorFlow model optimization?

easy

A. Remove less important weights to reduce model size

B. Convert model weights to lower precision

C. Add more layers to improve accuracy

D. Increase the batch size during training

Solution

1. Step 1: Understand pruning concept

   Pruning means removing parts of the model that contribute less, usually weights with small values.

2. Step 2: Identify pruning effect

   This reduces model size and speeds up inference by making the model simpler.

3. Final Answer:

   Remove less important weights to reduce model size -> Option A

4. Quick Check:

   Pruning = Remove less important weights [OK]

Hint: Pruning cuts small weights to shrink model size [OK]

Common Mistakes:

• Confusing pruning with quantization
• Thinking pruning adds layers
• Believing pruning changes batch size

2. Which of the following is the correct way to apply post-training quantization in TensorFlow Lite?

easy

A. converter.optimizations = [tf.lite.Optimize.DEFAULT]

B. model.compile(optimizer='adam', loss='mse')

C. tf.keras.layers.Dense(10, activation='relu')

D. model.fit(x_train, y_train, epochs=10)

Solution

1. Step 1: Recognize quantization syntax

   Post-training quantization in TensorFlow Lite uses the converter.optimizations property set to [tf.lite.Optimize.DEFAULT].

2. Step 2: Eliminate unrelated options

   Other options relate to training or model layers, not quantization.

3. Final Answer:

   converter.optimizations = [tf.lite.Optimize.DEFAULT] -> Option A

4. Quick Check:

   Quantization uses converter.optimizations = [tf.lite.Optimize.DEFAULT] [OK]

Hint: Quantization uses converter.optimizations setting [OK]

Common Mistakes:

• Confusing training code with quantization setup
• Using model.compile for quantization
• Applying quantization inside model.fit

3. What will be the output shape of the pruned model after applying TensorFlow Model Optimization pruning API on a model with input shape (None, 28, 28, 1) and a Dense layer with 10 units?

medium

A. (None, 28, 28, 10)

B. (None, 10)

C. (None, 28, 28, 1)

D. (10,)

Solution

1. Step 1: Understand pruning effect on shape

   Pruning removes weights but does not change the model architecture or output shape.

2. Step 2: Identify output shape of Dense layer

   A Dense layer with 10 units outputs shape (None, 10) regardless of pruning.

3. Final Answer:

   (None, 10) -> Option B

4. Quick Check:

   Pruning keeps output shape same = (None, 10) [OK]

Hint: Pruning doesn't change output shape, only weights [OK]

Common Mistakes:

• Thinking pruning changes output dimensions
• Confusing input shape with output shape
• Assuming pruning adds extra dimensions

4. You applied pruning to your TensorFlow model but noticed no size reduction after saving. What is the likely cause?

medium

A. You used quantization instead of pruning

B. Pruning increases model size by adding masks

C. Model was not stripped of pruning wrappers before saving

D. Model was saved in HDF5 format

Solution

1. Step 1: Understand pruning wrappers

   Pruning adds wrappers around layers that must be stripped before saving to reduce size.

2. Step 2: Identify saving step

   If wrappers are not stripped, the saved model keeps extra pruning info, so size doesn't reduce.

3. Final Answer:

   Model was not stripped of pruning wrappers before saving -> Option C

4. Quick Check:

   Strip pruning wrappers before saving = smaller model [OK]

Hint: Strip pruning wrappers before saving model [OK]

Common Mistakes:

• Assuming pruning always reduces size without stripping
• Confusing pruning with quantization effects
• Ignoring model saving format impact

5. You want to optimize a TensorFlow model for deployment on a mobile device with limited memory and CPU. Which combination of techniques is best to reduce model size and speed up inference without much accuracy loss?

hard

A. Convert model to float64 precision

B. Only increase training epochs without optimization

C. Use dropout layers and batch normalization

D. Apply pruning during training and then post-training quantization

Solution

1. Step 1: Identify optimization goals

   For mobile deployment, reducing size and speeding inference are key, while keeping accuracy.

2. Step 2: Choose suitable techniques

   Pruning removes unnecessary weights, quantization reduces number precision, both reduce size and speed up inference.

3. Step 3: Eliminate wrong options

   Increasing epochs doesn't optimize size; dropout and batch norm are for training regularization; float64 increases size.

4. Final Answer:

   Apply pruning during training and then post-training quantization -> Option D

5. Quick Check:

   Pruning + quantization = smaller, faster model [OK]

Hint: Combine pruning and quantization for best optimization [OK]

Common Mistakes:

• Ignoring quantization after pruning
• Confusing training tricks with optimization
• Using higher precision increases size