Mobile deployment (PyTorch Mobile) - ML Experiment: Train & Evaluate

Experiment - Mobile deployment (PyTorch Mobile)

Problem:You have trained a PyTorch image classification model on a desktop. Now you want to deploy it on a mobile device using PyTorch Mobile. The current model is large and slow to run on mobile.

Current Metrics:Model size: 50MB, Inference time on mobile: 2.5 seconds per image, Accuracy on test set: 88%

Issue:The model is too large and slow for mobile deployment, causing poor user experience.

Your Task

Reduce the model size and inference time on mobile while keeping test accuracy above 85%.

You must use PyTorch Mobile compatible techniques.

You cannot retrain the model from scratch.

You can only apply model optimization and conversion steps.

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
import torchvision.models as models

# Load pretrained model
model = models.resnet18(pretrained=True)
model.eval()

# Example input tensor
example_input = torch.randn(1, 3, 224, 224)

# Convert model to TorchScript
scripted_model = torch.jit.trace(model, example_input)

# Apply dynamic quantization
quantized_model = torch.quantization.quantize_dynamic(
    scripted_model, {torch.nn.Linear}, dtype=torch.qint8
)

# Save the quantized model for mobile
quantized_model.save('resnet18_quantized.pt')

# To test accuracy, run inference on test data (not shown here)
# Expected: model size reduced to ~10MB, inference time ~0.5s, accuracy ~86%

Converted the model to TorchScript using tracing for mobile compatibility.

Applied dynamic quantization to reduce model size and speed up inference.

Saved the optimized model in a mobile-friendly format.

Results Interpretation

Before Optimization: Model size 50MB, Inference time 2.5s, Accuracy 88%

After Optimization: Model size 10MB, Inference time 0.5s, Accuracy 86%

Using TorchScript and dynamic quantization can significantly reduce model size and speed up inference on mobile devices with only a small drop in accuracy.

Bonus Experiment

Try applying static quantization and compare its effect on model size, speed, and accuracy.

💡 Hint

Static quantization requires calibration with sample data before converting the model.

Practice

1. What is the main advantage of using PyTorch Mobile for AI models on phones?

easy

A. It converts models to TensorFlow format

B. It requires an internet connection to work

C. It runs models directly on the phone for faster response and privacy

D. It only supports training models on the phone

Solution

1. Step 1: Understand PyTorch Mobile purpose

   PyTorch Mobile allows running AI models directly on mobile devices without needing a server.

2. Step 2: Identify benefits of local model execution

   Running models locally improves speed and keeps data private since no internet is needed.

3. Final Answer:

   It runs models directly on the phone for faster response and privacy -> Option C

4. Quick Check:

   Local model execution = faster and private [OK]

Hint: Remember: Mobile deployment means running models on the device [OK]

Common Mistakes:

• Thinking PyTorch Mobile needs internet
• Confusing training with inference on mobile
• Mixing PyTorch Mobile with TensorFlow

2. Which PyTorch function is used to convert a model to TorchScript for mobile deployment?

easy

A. torch.jit.trace()

B. torch.save()

C. torch.load()

D. torch.nn.Module()

Solution

1. Step 1: Recall TorchScript conversion methods

   PyTorch uses torch.jit.trace() or torch.jit.script() to convert models to TorchScript.

2. Step 2: Identify correct function for conversion

   torch.jit.trace() records operations to create a TorchScript model for mobile use.

3. Final Answer:

   torch.jit.trace() -> Option A

4. Quick Check:

   Model conversion = torch.jit.trace() [OK]

Hint: Trace or script to convert model for mobile [OK]

Common Mistakes:

• Using torch.save() which only saves models
• Confusing torch.load() which loads models
• Using torch.nn.Module() which defines models

3. What will be the output type of this code snippet?

import torch
class MyModel(torch.nn.Module):
    def forward(self, x):
        return x * 2
model = MyModel()
traced_model = torch.jit.trace(model, torch.tensor([1]))
print(type(traced_model))

medium

A. <class 'torch.nn.Module'>

B. <class 'torch.jit.ScriptModule'>

C. <class 'torch.jit.TopLevelTracedModule'>

D. <class 'torch.jit.TracedModule'>

Solution

1. Step 1: Understand torch.jit.trace output type

   Tracing a model returns a TorchScript object of type ScriptModule in PyTorch.

2. Step 2: Check printed type

   Printing type(traced_model) shows <class 'torch.jit.ScriptModule'> indicating the traced script module.

3. Final Answer:

   <class 'torch.jit.ScriptModule'> -> Option B

4. Quick Check:

   Tracing returns ScriptModule [OK]

Hint: Trace returns ScriptModule type [OK]

Common Mistakes:

• Assuming output is TopLevelTracedModule
• Confusing traced model with normal nn.Module
• Choosing non-existent TracedModule class

4. Identify the error in this PyTorch Mobile model saving code:

import torch
model = torch.nn.Linear(2, 2)
traced = torch.jit.trace(model, torch.randn(2))
torch.save(traced, 'model.pt')
loaded = torch.jit.load('model.pt')
print(loaded(torch.randn(2)))

medium

A. torch.save() cannot save traced models

B. torch.jit.load() cannot load saved models

C. Input tensor shape for tracing is incorrect

D. Model must be scripted, not traced

Solution

1. Step 1: Identify the saving issue

   TorchScript models from torch.jit.trace() must use torch.jit.save(traced, 'model.pt'); torch.save() fails as ScriptModules are not picklable.

2. Step 2: Verify other components

   torch.randn(2) matches Linear(2,2) input shape; torch.jit.load() works after proper save.

3. Final Answer:

   torch.save() cannot save traced models -> Option A

4. Quick Check:

   TorchScript requires torch.jit.save() [OK]

Hint: torch.jit.save() for traced/ScriptModules [OK]

Common Mistakes:

• torch.save() doesn't pickle ScriptModules
• torch.jit.load() works correctly
• Traced models save with torch.jit.save()

5. You want to deploy a PyTorch model on Android. Which sequence correctly prepares the model for mobile use?

hard

A. Train model -> Convert to TorchScript with torch.save() -> Save as .pt -> Load in Android app

B. Train model -> Save as .pt -> Load directly in Android app -> Convert to TorchScript

C. Train model -> Convert to ONNX -> Save as .pt -> Load in Android app

D. Train model -> Convert to TorchScript with torch.jit.trace() -> Save as .pt -> Load in Android app

Solution

1. Step 1: Understand model conversion for mobile

   PyTorch Mobile requires models in TorchScript format, created by tracing or scripting.

2. Step 2: Identify correct conversion and saving steps

   Tracing with torch.jit.trace(), saving as .pt, then loading in Android app is the correct flow.

3. Final Answer:

   Train model -> Convert to TorchScript with torch.jit.trace() -> Save as .pt -> Load in Android app -> Option D

4. Quick Check:

   Trace -> save -> load for mobile deployment [OK]

Hint: Trace or script, save .pt, then load on mobile [OK]

Common Mistakes:

• Trying to load raw PyTorch model on mobile
• Using ONNX instead of TorchScript for PyTorch Mobile
• Saving before converting to TorchScript