Model optimization for serving (quantization, pruning) in MLOps - Step-by-Step Execution

Process Flow - Model optimization for serving (quantization, pruning)

Start with trained model

↓

Apply quantization

↓

Smaller model size

↓

Deploy optimized model

↓

Faster inference, less memory

This flow shows starting from a trained model, choosing quantization or pruning to reduce size and improve serving speed, then deploying the optimized model.

Execution Sample

MLOps

import tensorflow as tf
model = tf.keras.models.load_model('model.h5')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_model = converter.convert()
with open('model_quantized.tflite', 'wb') as f:
    f.write(quantized_model)

This code loads a trained model, applies quantization to reduce size, and saves the optimized model for serving.

Process Table

Step	Action	Input Model Size (MB)	Output Model Size (MB)	Effect
1	Load trained model	50	50	Model loaded, no change
2	Apply quantization	50	12	Model size reduced by ~76%
3	Save quantized model	12	12	Optimized model saved
4	Deploy model	12	12	Ready for faster serving

💡 Model optimized and deployed with reduced size for efficient serving

Status Tracker

Variable	Start	After Quantization	After Saving	Final
model_size_MB	50	12	12	12
model_state	trained	quantized	saved	deployed

Key Moments - 3 Insights

Why does the model size drop significantly after quantization?

Quantization reduces the precision of numbers in the model (e.g., from 32-bit floats to 8-bit integers), which shrinks the model size as shown in step 2 of the execution table.

Does pruning always reduce model size as much as quantization?

No, pruning removes less important connections but may not reduce size as much as quantization. The execution table focuses on quantization size change for clarity.

Why save the model after optimization if size is already reduced?

Saving stores the optimized model in a deployable format, ensuring the smaller size and changes persist, as shown in step 3.

Visual Quiz - 3 Questions

Look at the execution table, what is the model size after quantization?

A 12 MB

B 50 MB

C 25 MB

D 5 MB

💡 Hint

Check the 'Output Model Size (MB)' column at step 2 in the execution table.

At which step is the model ready for faster serving?

A Step 1

B Step 2

C Step 4

D Step 3

💡 Hint

Look at the 'Effect' column describing deployment readiness in the execution table.

If pruning was applied instead of quantization, what would likely change in the execution table?

A Model size would increase

B Model size reduction might be less drastic

C Model size would be zero

D No change in model size

💡 Hint

Refer to the key moments section explaining pruning effects compared to quantization.

Concept Snapshot

Model optimization for serving:
- Start with trained model
- Choose quantization (reduce number precision) or pruning (remove connections)
- Apply chosen method to reduce model size
- Save and deploy optimized model
- Result: faster inference and less memory use

Full Transcript

This visual execution shows how a trained machine learning model is optimized for serving by applying quantization or pruning. First, the trained model is loaded with its original size. Then quantization is applied, which reduces the model size significantly by lowering number precision. The optimized model is saved to keep the changes. Finally, the smaller model is deployed for faster inference and lower memory use. Key points include understanding why quantization reduces size more than pruning and the importance of saving the optimized model. The execution table tracks model size and state at each step, helping beginners see the impact of each action clearly.