TorchServe setup in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding TorchServe Model Archiving

What is the primary purpose of the torch-model-archiver tool in TorchServe?

A To train a PyTorch model using distributed GPUs

B To visualize the model architecture graphically

C To convert a PyTorch model into TensorFlow format

D To package a trained PyTorch model and its dependencies into a .mar file for serving

💡 Hint

Think about how TorchServe loads models for inference.

The torch-model-archiver packages the model and related files into a single archive (.mar) that TorchServe can load for serving predictions.

❓ Predict Output

intermediate

Predicting Output of TorchServe Registration Command

What will be the output message when running this command to register a model with TorchServe?

torchserve --start --model-store model_store --models mymodel=mymodel.mar

A TorchServe started successfully and model 'mymodel' is registered and ready for inference.

B Error: Model store directory 'model_store' not found.

C SyntaxError: invalid command line option.

D TorchServe started but model 'mymodel' failed to load due to missing handler.

💡 Hint

Assume the model store and .mar file exist and are correct.

If the model store directory and .mar file exist and are valid, TorchServe starts and registers the model successfully.

❓ Model Choice

advanced

Choosing the Correct Handler for Custom Model

You have a PyTorch model that takes two inputs and returns a dictionary of outputs. Which handler type should you use in TorchServe to serve this model?

A Use the default <code>image_classifier</code> handler

B Use the <code>text_classifier</code> handler

C Create a custom handler by subclassing <code>BaseHandler</code>

D Use the <code>object_detector</code> handler

💡 Hint

Default handlers expect specific input/output formats.

When the model input/output does not match default handlers, a custom handler subclassing BaseHandler is needed to process inputs and outputs properly.

❓ Hyperparameter

advanced

Configuring Batch Size in TorchServe

Which configuration file and parameter should you modify to change the batch size for inference requests in TorchServe?

A Modify <code>model-config.yaml</code> and set <code>max_batch_size</code> under the model entry

B Modify <code>config.properties</code> and set <code>batch_size</code>

C Modify <code>handler.py</code> to manually batch inputs

D Modify <code>torchserve.conf</code> and set <code>max_batch_size</code>

💡 Hint

Batch size is usually set per model in a YAML config.

The model-config.yaml file defines model-specific parameters including max_batch_size for inference batching.

❓ Metrics

expert

Interpreting TorchServe Metrics Output

After running TorchServe with metrics enabled, you see this output snippet:

{"model_name": "mymodel", "inference_count": 1000, "average_latency": 25.3}

What does the average_latency value represent?

A The average time in milliseconds taken to load the model at startup

B The average time in milliseconds taken to process each inference request

C The average time in seconds between two consecutive inference requests

D The average time in microseconds taken to serialize the model

💡 Hint

Latency usually measures processing time per request.

The average_latency metric shows the average time in milliseconds TorchServe takes to process each inference request for the model.