Model optimization for serving (quantization, pruning) in MLOps - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to apply post-training quantization using TensorFlow Lite.

MLOps

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [[1]]
tflite_model = converter.convert()

Drag options to blanks, or click blank then click option'

Atf.lite.Optimize.DEFAULT

Btf.lite.Quantize.DEFAULT

Ctf.lite.Optimize.QUANTIZE

Dtf.lite.Optimize.NONE

Attempts:

3 left

💡 Hint

Common Mistakes

Using a non-existent flag like tf.lite.Quantize.DEFAULT

Using NONE disables optimization, so no quantization happens

The correct optimization flag for post-training quantization in TensorFlow Lite is tf.lite.Optimize.DEFAULT. This enables default optimizations including quantization.

2fill in blank

medium

Complete the code to prune a Keras model with 50% sparsity.

MLOps

import tensorflow_model_optimization as tfmot
prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude
pruning_params = {'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(initial_sparsity=0.0, final_sparsity=[1], begin_step=0, end_step=1000)}
model = prune_low_magnitude(original_model, **pruning_params)

Drag options to blanks, or click blank then click option'

A50

B0.5

C0.05

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Using 50 instead of 0.5 causes errors because sparsity must be between 0 and 1

Using 5 or 0.05 are incorrect values for 50% sparsity

Sparsity is expressed as a fraction between 0 and 1. For 50% sparsity, use 0.5.

3fill in blank

hard

Fix the error in the pruning callback setup to correctly update pruning steps during training.

MLOps

callbacks = [tfmot.sparsity.keras.UpdatePruningStep(), [1]]
model.fit(train_data, epochs=10, callbacks=callbacks)

Drag options to blanks, or click blank then click option'

Atf.keras.callbacks.EarlyStopping()

Btfmot.sparsity.keras.PruningCallback()

Ctfmot.sparsity.keras.PruningSummaries(log_dir)

Dtf.keras.callbacks.ModelCheckpoint()

Attempts:

3 left

💡 Hint

Common Mistakes

Using unrelated callbacks like EarlyStopping or ModelCheckpoint

Using a non-existent PruningCallback

The correct callback to add alongside UpdatePruningStep() is PruningSummaries to log pruning summaries during training.

4fill in blank

hard

Fill both blanks to create a TensorFlow Lite converter that applies quantization and sets the supported ops.

MLOps

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [[1]]
converter.target_spec.supported_ops = [[2]]
tflite_model = converter.convert()

Drag options to blanks, or click blank then click option'

Atf.lite.Optimize.DEFAULT

Btf.lite.OpsSet.TFLITE_BUILTINS_INT8

Ctf.lite.OpsSet.TFLITE_BUILTINS

Dtf.lite.Optimize.NONE

Attempts:

3 left

💡 Hint

Common Mistakes

Using NONE disables optimization.

Using TFLITE_BUILTINS without INT8 does not fully enable quantized ops.

Use tf.lite.Optimize.DEFAULT to enable quantization and tf.lite.OpsSet.TFLITE_BUILTINS_INT8 to specify integer-only ops for full quantization support.

5fill in blank

hard

Fill all three blanks to create a pruning schedule with 20% initial sparsity, 80% final sparsity, and pruning ending at step 2000.

MLOps

pruning_params = {
  'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(initial_sparsity=[1], final_sparsity=[2], begin_step=0, end_step=[3])
}

Drag options to blanks, or click blank then click option'

A0.2

B0.8

C2000

D1000

Attempts:

3 left

💡 Hint

Common Mistakes

Swapping initial and final sparsity values.

Using incorrect end step values like 1000 instead of 2000.

The initial sparsity is 20% (0.2), final sparsity is 80% (0.8), and pruning ends at step 2000.

Practice

1. What is the main goal of quantization in model optimization for serving?

easy

A. Increase the size of the model for better performance

B. Reduce the precision of numbers to make the model smaller and faster

C. Add more neurons to improve accuracy

D. Remove entire layers from the model to simplify it

Solution

1. Step 1: Understand quantization purpose

   Quantization reduces the number precision (like from 32-bit to 8-bit) to save memory and speed up computation.

2. Step 2: Compare options

   Removing layers is pruning, adding neurons increases size, increasing size is opposite of optimization.

3. Final Answer:

   Reduce the precision of numbers to make the model smaller and faster -> Option B

4. Quick Check:

   Quantization = Reduce precision [OK]

Hint: Quantization means lowering number precision to save space [OK]

Common Mistakes:

• Confusing pruning with quantization
• Thinking quantization adds complexity
• Believing quantization increases model size

2. Which of the following is the correct syntax to apply pruning using TensorFlow Model Optimization API in Python?

easy

A. pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule)

B. pruned_model = tf.prune_low_magnitude(model, schedule=pruning_schedule)

C. pruned_model = tfmot.prune_low_magnitude(model, pruning_schedule=pruning_schedule)

D. pruned_model = tfmot.sparsity.prune_low_magnitude(model, pruning_schedule)

Solution

1. Step 1: Recall TensorFlow pruning API structure

   The pruning function is under tfmot.sparsity.keras and requires the pruning_schedule argument.

2. Step 2: Check syntax correctness

   pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) matches the correct full path and argument names. Others miss parts or have wrong argument names.

3. Final Answer:

   pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) -> Option A

4. Quick Check:

   Correct pruning syntax = pruned_model = tfmot.sparsity.keras.prune_low_magnitude(model, pruning_schedule=pruning_schedule) [OK]

Hint: TensorFlow pruning is under tfmot.sparsity.keras with pruning_schedule [OK]

Common Mistakes:

• Omitting 'keras' in the API path
• Using wrong argument names
• Calling pruning directly from tf module

3. Given the following PyTorch code snippet for quantization, what will be the output type of the model's weights after applying dynamic quantization?

import torch
import torch.nn as nn

model = nn.Linear(10, 5)
quantized_model = torch.quantization.quantize_dynamic(model, {nn.Linear}, dtype=torch.qint8)
print(type(quantized_model.weight()))

medium

A. TypeError: 'weight' is not callable

B.

C. AttributeError: 'Linear' object has no attribute 'weight'

D.

Solution

1. Step 1: Analyze dynamic quantization effect

   torch.quantization.quantize_dynamic converts nn.Linear to torch.nn.quantized.dynamic.Linear, where weight is a method returning dequantized weights as torch.Tensor.

2. Step 2: Trace the print statement

   quantized_model.weight() succeeds, returning a torch.Tensor (fp32 dequantized), so print(type(...)) outputs <class 'torch.Tensor'>.

3. Final Answer:

   <class 'torch.Tensor'> -> Option D

4. Quick Check:

   Dynamic quant: weight() returns Tensor [OK]

Hint: Dynamic quantization makes weight() callable returning Tensor [OK]

Common Mistakes:

• Thinking weight remains non-callable attribute like original Linear
• Confusing quantized_model type with weight type
• Expecting error on quantized model weight access

4. You tried pruning a TensorFlow model but got an error: AttributeError: module 'tensorflow_model_optimization' has no attribute 'sparsity'. What is the most likely cause?

medium

A. The tensorflow_model_optimization package is not installed

B. You used the wrong pruning schedule argument

C. You forgot to import tensorflow_model_optimization as tfmot

D. Pruning is not supported in TensorFlow

Solution

1. Step 1: Understand the error message

   The error says the module has no attribute 'sparsity', which usually means the package is missing or outdated.

2. Step 2: Check common causes

   If the package is not installed, Python cannot find the 'sparsity' submodule. Importing incorrectly or wrong argument causes different errors.

3. Final Answer:

   The tensorflow_model_optimization package is not installed -> Option A

4. Quick Check:

   Missing package = AttributeError [OK]

Hint: Missing package causes AttributeError on submodules [OK]

Common Mistakes:

• Assuming import alias causes error
• Blaming pruning schedule argument
• Thinking pruning unsupported in TensorFlow

5. You want to optimize a large deep learning model for mobile deployment by combining pruning and quantization. Which sequence of steps is best to minimize model size and maintain accuracy?

hard

A. Apply quantization first, then prune the model to remove weights

B. Train the model with quantization-aware training, then prune after deployment

C. First prune the model to remove unimportant weights, then apply quantization to reduce number precision

D. Only prune the model; quantization is not compatible with pruning

Solution

1. Step 1: Understand pruning and quantization order

   Pruning removes unimportant weights first, reducing model size and complexity.

2. Step 2: Apply quantization after pruning

   Quantization then reduces number precision on the smaller pruned model, further shrinking size and speeding inference.

3. Final Answer:

   First prune the model to remove unimportant weights, then apply quantization to reduce number precision -> Option C

4. Quick Check:

   Prune first, then quantize = First prune the model to remove unimportant weights, then apply quantization to reduce number precision [OK]

Hint: Prune first to shrink, then quantize to compress numbers [OK]

Common Mistakes:

• Quantizing before pruning reduces pruning effectiveness
• Thinking pruning and quantization cannot be combined
• Pruning after deployment is too late