Mobile deployment (PyTorch Mobile) - ML Experiment: Train & Evaluate

Experiment - Mobile deployment (PyTorch Mobile)

Problem:You have trained a PyTorch image classification model on a desktop. Now you want to deploy it on a mobile device using PyTorch Mobile. The current model is large and slow to run on mobile.

Current Metrics:Model size: 50MB, Inference time on mobile: 2.5 seconds per image, Accuracy on test set: 88%

Issue:The model is too large and slow for mobile deployment, causing poor user experience.

Your Task

Reduce the model size and inference time on mobile while keeping test accuracy above 85%.

You must use PyTorch Mobile compatible techniques.

You cannot retrain the model from scratch.

You can only apply model optimization and conversion steps.

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
import torchvision.models as models

# Load pretrained model
model = models.resnet18(pretrained=True)
model.eval()

# Example input tensor
example_input = torch.randn(1, 3, 224, 224)

# Convert model to TorchScript
scripted_model = torch.jit.trace(model, example_input)

# Apply dynamic quantization
quantized_model = torch.quantization.quantize_dynamic(
    scripted_model, {torch.nn.Linear}, dtype=torch.qint8
)

# Save the quantized model for mobile
quantized_model.save('resnet18_quantized.pt')

# To test accuracy, run inference on test data (not shown here)
# Expected: model size reduced to ~10MB, inference time ~0.5s, accuracy ~86%

Converted the model to TorchScript using tracing for mobile compatibility.

Applied dynamic quantization to reduce model size and speed up inference.

Saved the optimized model in a mobile-friendly format.

Results Interpretation

Before Optimization: Model size 50MB, Inference time 2.5s, Accuracy 88%

After Optimization: Model size 10MB, Inference time 0.5s, Accuracy 86%

Using TorchScript and dynamic quantization can significantly reduce model size and speed up inference on mobile devices with only a small drop in accuracy.

Bonus Experiment

Try applying static quantization and compare its effect on model size, speed, and accuracy.

💡 Hint

Static quantization requires calibration with sample data before converting the model.