REST API inference in PyTorch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Flask REST API inference code?

Consider this simple Flask app that loads a PyTorch model and returns predictions for input data sent as JSON. What will be the JSON response when sending {"input": [1.0, 2.0, 3.0]}?

PyTorch

from flask import Flask, request, jsonify
import torch
import torch.nn as nn

app = Flask(__name__)

class SimpleModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(3, 1)
    def forward(self, x):
        return self.linear(x)

model = SimpleModel()
model.linear.weight.data.fill_(1.0)
model.linear.bias.data.fill_(0.0)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    x = torch.tensor(data['input'], dtype=torch.float32)
    with torch.no_grad():
        output = model(x).item()
    return jsonify({'prediction': output})

A {"prediction": 1.0}

B {"prediction": 3.0}

C {"prediction": 0.0}

D {"prediction": 6.0}

💡 Hint

Remember the linear layer sums weighted inputs plus bias. Weights are all 1, bias is 0.

The linear layer has weights all set to 1 and bias 0. Input vector is [1.0, 2.0, 3.0]. The output is sum(1*1 + 1*2 + 1*3) + 0 = 6.0.

❓ Model Choice

intermediate

Which PyTorch model is best suited for REST API inference on image classification?

You want to deploy a REST API that receives images and returns predicted labels. Which model architecture below is most appropriate for this task?

A A linear regression model

B A recurrent neural network (RNN) like LSTM

C A convolutional neural network (CNN) like ResNet

D A k-means clustering model

💡 Hint

Think about which model type handles image data well.

CNNs like ResNet are designed to process images and extract spatial features, making them ideal for image classification tasks in REST APIs.

❓ Hyperparameter

advanced

Which batch size is best for fast REST API inference with PyTorch?

You deploy a REST API for inference. To optimize latency and throughput, which batch size should you choose?

A Batch size of 1 for lowest latency per request

B Batch size of 128 to maximize GPU utilization

C Batch size of 0 to disable batching

D Batch size of 1000 to process many requests at once

💡 Hint

Consider that REST API requests usually come one at a time and latency matters.

Batch size 1 processes each request immediately, minimizing latency. Large batch sizes increase throughput but add delay waiting for batch to fill.

🔧 Debug

advanced

Why does this PyTorch REST API inference code raise a RuntimeError?

Given this code snippet, why does the server raise a RuntimeError: "Trying to backward through the graph during inference"?

PyTorch

from flask import Flask, request, jsonify
import torch
import torch.nn as nn

app = Flask(__name__)

model = nn.Linear(2, 1)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    x = torch.tensor(data['input'], dtype=torch.float32)
    with torch.no_grad():
        output = model(x)
    return jsonify({'prediction': output.item()})

A Flask cannot handle POST requests without JSON schema validation

B The code is missing torch.no_grad() context, so autograd tries to track operations

C The input tensor x has wrong dtype causing runtime error

D The model is not in eval() mode causing training behavior

💡 Hint

Inference should not track gradients to save memory and avoid errors.

Without torch.no_grad(), PyTorch tracks operations for gradients. During inference, this can cause errors if backward is called or memory issues.

❓ Metrics

expert

Which metric is best to monitor REST API inference quality for a multi-class classification model?

You deploy a REST API serving a multi-class classifier. Which metric below best measures the model's prediction quality on live data?

A Accuracy - percentage of correct predictions

B Mean Squared Error (MSE)

C Silhouette Score

D BLEU Score

💡 Hint

Think about classification metrics that reflect correct label predictions.

Accuracy directly measures the proportion of correct predictions in classification tasks, making it suitable for monitoring REST API inference quality.