Model packaging (.mar files) in PyTorch - ML Experiment: Train & Evaluate

Experiment - Model packaging (.mar files)

Problem:You have trained a PyTorch image classification model but it is only saved as a .pt file. You want to package it into a .mar file for easy deployment with TorchServe.

Current Metrics:Model accuracy on test data: 85%. Model saved as model.pt file.

Issue:The model is not packaged for deployment. TorchServe requires a .mar file which bundles the model and handler code.

Your Task

Package the existing PyTorch model into a .mar file that TorchServe can use for deployment.

You must use torch-model-archiver tool.

You cannot retrain the model.

You must include a custom handler for preprocessing and postprocessing.

Hint 1

Hint 2

Hint 3

Solution

PyTorch

import torch
import torchvision.models as models

# Assume model.pt is already saved
# handler.py content:
handler_code = '''
from ts.torch_handler.base_handler import BaseHandler
import torch
from torchvision import transforms
from PIL import Image
import io

class CustomHandler(BaseHandler):
    def preprocess(self, data):
        image = data[0].get("data") or data[0].get("body")
        image = Image.open(io.BytesIO(image)).convert("RGB")
        transform = transforms.Compose([
            transforms.Resize(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])
        ])
        return transform(image).unsqueeze(0)

    def inference(self, input_tensor):
        with torch.no_grad():
            output = self.model(input_tensor)
        return output

    def postprocess(self, inference_output):
        probabilities = torch.nn.functional.softmax(inference_output[0], dim=0)
        return probabilities.tolist()
'''

with open('handler.py', 'w') as f:
    f.write(handler_code)

# Command to create .mar file (run in shell):
# torch-model-archiver --model-name my_model --version 1.0 --serialized-file model.pt --handler handler.py --export-path model_store --force

# After running above command, the my_model.mar file will be in model_store folder.

# To verify loading the model in TorchServe (example):
# torchserve --start --model-store model_store --models my_model=my_model.mar

Created a custom handler.py file with preprocess, inference, and postprocess methods.

Used torch-model-archiver CLI to package model.pt and handler.py into my_model.mar.

Prepared the .mar file for deployment with TorchServe.

Results Interpretation

Before: Model saved only as model.pt file, not deployable directly.

After: Model packaged as my_model.mar file including custom handler, ready for TorchServe deployment.

Packaging a PyTorch model into a .mar file bundles the model and processing code, making deployment easier and standardized with TorchServe.

Bonus Experiment

Try packaging the model with a default handler instead of a custom one and compare deployment ease.

💡 Hint

Use the built-in image_classifier handler provided by TorchServe and see if it fits your preprocessing needs.

Practice

1. What is the main purpose of packaging a PyTorch model into a .mar file?

easy

A. To visualize the model architecture

B. To convert the model into a different training format

C. To compress the model for storage only

D. To bundle the model and code for easy deployment with TorchServe

Solution

1. Step 1: Understand what a .mar file contains

   A .mar file bundles the trained model and the handler code needed to serve it.

2. Step 2: Identify the use case of .mar files

   These files are used by TorchServe to deploy models as APIs easily.

3. Final Answer:

   To bundle the model and code for easy deployment with TorchServe -> Option D

4. Quick Check:

   Model packaging = deployment bundle [OK]

Hint: Think deployment: .mar bundles model + code [OK]

Common Mistakes:

• Confusing .mar with just compression
• Thinking .mar changes training format
• Assuming .mar is for visualization

2. Which command correctly creates a .mar file using torch-model-archiver?

easy

A. torch-model-archiver --model-name mymodel --model-file model.pth --handler handler.py --export-path model_store

B. torch-model-archiver --model-name mymodel --model-file model.pth --handler handler.txt --export-path model_store

C. torch-model-archiver --model-name mymodel --model-file model.txt --handler handler.py --export-path model_store

D. torch-model-archiver --model-name mymodel --model-file model.py --handler handler.py --export-path model_store

Solution

1. Step 1: Identify correct model file type

   The model file should be a PyTorch model file like .pth, not a Python script or text file.

2. Step 2: Confirm handler file type

   The handler must be a Python script .py, not a text file.

3. Final Answer:

   torch-model-archiver --model-name mymodel --model-file model.pth --handler handler.py --export-path model_store -> Option A

4. Quick Check:

   Model file = .pth, handler = .py [OK]

Hint: Model file is .pth, handler is .py script [OK]

Common Mistakes:

• Using .py file as model file
• Using .txt as handler file
• Mixing file extensions incorrectly

3. Given this command:

torch-model-archiver --model-name resnet --model-file resnet.pth --handler image_classifier.py --export-path model_store

What will be the output?

medium

A. A Python script named resnet.py created

B. An error because resnet.pth is not a valid model file

C. A file named resnet.mar created in model_store directory

D. A folder named resnet.mar created

Solution

1. Step 1: Understand torch-model-archiver output

   The command packages the model and handler into a single .mar file.

2. Step 2: Check output location and file name

   The .mar file is named after the model name and saved in the export path.

3. Final Answer:

   A file named resnet.mar created in model_store directory -> Option C

4. Quick Check:

   .mar file = model + handler bundle [OK]

Hint: Output is a .mar file named after model [OK]

Common Mistakes:

• Expecting a Python script output
• Thinking .mar is a folder
• Assuming error due to .pth file

4. You ran this command:

torch-model-archiver --model-name mymodel --model-file model.pth --handler handler.py --export-path model_store

But got an error: Handler file not found. What is the likely cause?

medium

A. The handler.py file is missing or path is incorrect

B. The model.pth file is corrupted

C. The export path model_store does not exist

D. The model name contains invalid characters

Solution

1. Step 1: Analyze the error message

   The error says handler file not found, so the problem is with the handler.py file location or name.

2. Step 2: Check other options

   Model file corruption or export path issues would cause different errors, not handler missing.

3. Final Answer:

   The handler.py file is missing or path is incorrect -> Option A

4. Quick Check:

   Error message = handler file missing [OK]

Hint: Check handler file path if 'Handler file not found' error occurs [OK]

Common Mistakes:

• Blaming model file for handler error
• Ignoring file path correctness
• Assuming export path causes handler error

5. You want to package two different PyTorch models with custom handlers into .mar files for TorchServe. Which approach correctly packages both models for deployment?

hard

A. Package one model as .mar and copy the second model file into the model_store folder without packaging

B. Run torch-model-archiver twice, each time with the respective model file, handler, and unique model name, exporting to the same model_store folder

C. Combine both models into one .pth file and use a single handler to package once

D. Use torch-model-archiver once with both model files and handlers listed separated by commas

Solution

1. Step 1: Understand packaging multiple models

   Each model requires its own .mar file with its specific handler and unique model name.

2. Step 2: Confirm correct packaging method

   Running torch-model-archiver twice with different parameters and exporting to the same folder is the correct way.

3. Final Answer:

   Run torch-model-archiver twice, each time with the respective model file, handler, and unique model name, exporting to the same model_store folder -> Option B

4. Quick Check:

   One .mar per model + handler [OK]

Hint: Package each model separately with unique names [OK]

Common Mistakes:

• Trying to combine models in one .mar file
• Copying model files without packaging
• Passing multiple files in one command