Model optimization for serving (quantization, pruning) in MLOps - Step-by-Step Execution

Process Flow - Model optimization for serving (quantization, pruning)

Start with trained model

↓

Apply quantization

↓

Smaller model size

↓

Deploy optimized model

↓

Faster inference, less memory

This flow shows starting from a trained model, choosing quantization or pruning to reduce size and improve serving speed, then deploying the optimized model.

Execution Sample

MLOps

import tensorflow as tf
model = tf.keras.models.load_model('model.h5')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_model = converter.convert()
with open('model_quantized.tflite', 'wb') as f:
    f.write(quantized_model)

This code loads a trained model, applies quantization to reduce size, and saves the optimized model for serving.

Process Table

Step	Action	Input Model Size (MB)	Output Model Size (MB)	Effect
1	Load trained model	50	50	Model loaded, no change
2	Apply quantization	50	12	Model size reduced by ~76%
3	Save quantized model	12	12	Optimized model saved
4	Deploy model	12	12	Ready for faster serving

💡 Model optimized and deployed with reduced size for efficient serving

Status Tracker

Variable	Start	After Quantization	After Saving	Final
model_size_MB	50	12	12	12
model_state	trained	quantized	saved	deployed

Key Moments - 3 Insights

Why does the model size drop significantly after quantization?

Quantization reduces the precision of numbers in the model (e.g., from 32-bit floats to 8-bit integers), which shrinks the model size as shown in step 2 of the execution table.

Does pruning always reduce model size as much as quantization?

No, pruning removes less important connections but may not reduce size as much as quantization. The execution table focuses on quantization size change for clarity.

Why save the model after optimization if size is already reduced?

Saving stores the optimized model in a deployable format, ensuring the smaller size and changes persist, as shown in step 3.

Visual Quiz - 3 Questions

Look at the execution table, what is the model size after quantization?

A 12 MB

B 50 MB

C 25 MB

D 5 MB

💡 Hint

Check the 'Output Model Size (MB)' column at step 2 in the execution table.

At which step is the model ready for faster serving?

A Step 1

B Step 2

C Step 4

D Step 3

💡 Hint

Look at the 'Effect' column describing deployment readiness in the execution table.

If pruning was applied instead of quantization, what would likely change in the execution table?

A Model size would increase

B Model size reduction might be less drastic

C Model size would be zero

D No change in model size

💡 Hint

Refer to the key moments section explaining pruning effects compared to quantization.

Concept Snapshot

Model optimization for serving:
- Start with trained model
- Choose quantization (reduce number precision) or pruning (remove connections)
- Apply chosen method to reduce model size
- Save and deploy optimized model
- Result: faster inference and less memory use

Full Transcript

This visual execution shows how a trained machine learning model is optimized for serving by applying quantization or pruning. First, the trained model is loaded with its original size. Then quantization is applied, which reduces the model size significantly by lowering number precision. The optimized model is saved to keep the changes. Finally, the smaller model is deployed for faster inference and lower memory use. Key points include understanding why quantization reduces size more than pruning and the importance of saving the optimized model. The execution table tracks model size and state at each step, helping beginners see the impact of each action clearly.

Practice

1. What is the main goal of quantization in model optimization for serving?

easy

A. Increase the size of the model for better performance

B. Reduce the precision of numbers to make the model smaller and faster

C. Add more neurons to improve accuracy

D. Remove entire layers from the model to simplify it

Solution

1. Step 1: Understand quantization purpose

   Quantization reduces the number precision (like from 32-bit to 8-bit) to save memory and speed up computation.

2. Step 2: Compare options

   Removing layers is pruning, adding neurons increases size, increasing size is opposite of optimization.

3. Final Answer:

   Reduce the precision of numbers to make the model smaller and faster -> Option B

4. Quick Check:

   Quantization = Reduce precision [OK]

Hint: Quantization means lowering number precision to save space [OK]

Common Mistakes:

• Confusing pruning with quantization
• Thinking quantization adds complexity
• Believing quantization increases model size

2. Which of the following is the correct syntax to apply pruning using TensorFlow Model Optimization API in Python?

easy

A. pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule)

B. pruned_model = tf.prune_low_magnitude(model, schedule=pruning_schedule)

C. pruned_model = tfmot.prune_low_magnitude(model, pruning_schedule=pruning_schedule)

D. pruned_model = tfmot.sparsity.prune_low_magnitude(model, pruning_schedule)

Solution

1. Step 1: Recall TensorFlow pruning API structure

   The pruning function is under tfmot.sparsity.keras and requires the pruning_schedule argument.

2. Step 2: Check syntax correctness

   pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) matches the correct full path and argument names. Others miss parts or have wrong argument names.

3. Final Answer:

   pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) -> Option A

4. Quick Check:

   Correct pruning syntax = pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) [OK]

Hint: TensorFlow pruning is under tfmot.sparsity.keras with pruning_schedule [OK]

Common Mistakes:

• Omitting 'keras' in the API path
• Using wrong argument names
• Calling pruning directly from tf module

3. Given the following PyTorch code snippet for quantization, what will be the output type of the model's weights after applying dynamic quantization?

import torch
import torch.nn as nn

model = nn.Linear(10, 5)
quantized_model = torch.quantization.quantize_dynamic(model, {nn.Linear}, dtype=torch.qint8)
print(type(quantized_model.weight()))

medium

A. TypeError: 'weight' is not callable

B.

C. AttributeError: 'Linear' object has no attribute 'weight'

D.

Solution

1. Step 1: Analyze dynamic quantization effect

   torch.quantization.quantize_dynamic converts nn.Linear to torch.nn.quantized.dynamic.Linear, where weight is a method returning dequantized weights as torch.Tensor.

2. Step 2: Trace the print statement

   quantized_model.weight() succeeds, returning a torch.Tensor (fp32 dequantized), so print(type(...)) outputs <class 'torch.Tensor'>.

3. Final Answer:

   <class 'torch.Tensor'> -> Option D

4. Quick Check:

   Dynamic quant: weight() returns Tensor [OK]

Hint: Dynamic quantization makes weight() callable returning Tensor [OK]

Common Mistakes:

• Thinking weight remains non-callable attribute like original Linear
• Confusing quantized_model type with weight type
• Expecting error on quantized model weight access

4. You tried pruning a TensorFlow model but got an error: AttributeError: module 'tensorflow_model_optimization' has no attribute 'sparsity'. What is the most likely cause?

medium

A. The tensorflow_model_optimization package is not installed

B. You used the wrong pruning schedule argument

C. You forgot to import tensorflow_model_optimization as tfmot

D. Pruning is not supported in TensorFlow

Solution

1. Step 1: Understand the error message

   The error says the module has no attribute 'sparsity', which usually means the package is missing or outdated.

2. Step 2: Check common causes

   If the package is not installed, Python cannot find the 'sparsity' submodule. Importing incorrectly or wrong argument causes different errors.

3. Final Answer:

   The tensorflow_model_optimization package is not installed -> Option A

4. Quick Check:

   Missing package = AttributeError [OK]

Hint: Missing package causes AttributeError on submodules [OK]

Common Mistakes:

• Assuming import alias causes error
• Blaming pruning schedule argument
• Thinking pruning unsupported in TensorFlow

5. You want to optimize a large deep learning model for mobile deployment by combining pruning and quantization. Which sequence of steps is best to minimize model size and maintain accuracy?

hard

A. Apply quantization first, then prune the model to remove weights

B. Train the model with quantization-aware training, then prune after deployment

C. First prune the model to remove unimportant weights, then apply quantization to reduce number precision

D. Only prune the model; quantization is not compatible with pruning

Solution

1. Step 1: Understand pruning and quantization order

   Pruning removes unimportant weights first, reducing model size and complexity.

2. Step 2: Apply quantization after pruning

   Quantization then reduces number precision on the smaller pruned model, further shrinking size and speeding inference.

3. Final Answer:

   First prune the model to remove unimportant weights, then apply quantization to reduce number precision -> Option C

4. Quick Check:

   Prune first, then quantize = First prune the model to remove unimportant weights, then apply quantization to reduce number precision [OK]

Hint: Prune first to shrink, then quantize to compress numbers [OK]

Common Mistakes:

• Quantizing before pruning reduces pruning effectiveness
• Thinking pruning and quantization cannot be combined
• Pruning after deployment is too late