Model serving (TF Serving) in TensorFlow - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary role of TensorFlow Serving?

TensorFlow Serving is used in production environments. What is its main purpose?

A To preprocess data before training a model

B To train machine learning models faster using GPUs

C To visualize model training metrics in real-time

D To deploy and serve machine learning models efficiently with version management

💡 Hint

Think about what happens after a model is trained and needs to be used by applications.

TensorFlow Serving is designed to deploy and serve models in production, handling multiple versions and providing a stable API for inference.

❓ Predict Output

intermediate

What output does this TensorFlow Serving REST API request produce?

Given a model served at localhost:8501 with signature 'serving_default', what is the output of this request?

TensorFlow

import requests
import json

url = 'http://localhost:8501/v1/models/my_model:predict'
data = {"instances": [[1.0, 2.0, 3.0]]}
response = requests.post(url, json=data)
print(response.json())

A {"predictions": [[0.5]]}

B {"error": "Model not found"}

C {"predictions": [[1.0, 2.0, 3.0]]}

D requests.exceptions.ConnectionError

💡 Hint

Assume the model is loaded and expects 3 input features, outputting a single prediction.

The request sends input data to the model and receives predictions. Option A shows a typical prediction output format.

❓ Model Choice

advanced

Which model export format is required for TensorFlow Serving?

You want to serve a TensorFlow model using TensorFlow Serving. Which export format must you use?

A SavedModel format with signature definitions

B HDF5 (.h5) format

C Pickle serialized model

D ONNX format

💡 Hint

TensorFlow Serving requires a format that includes computation graph and weights.

TensorFlow Serving uses the SavedModel format which contains the model graph, weights, and signatures for serving.

❓ Hyperparameter

advanced

Which TensorFlow Serving configuration parameter controls model version policy?

You want TensorFlow Serving to keep only the latest 3 versions of a model loaded. Which config parameter do you set?

A max_num_versions: 3

B version_limit: 3

C model_version_policy { latest { num_versions: 3 } }

D keep_versions: 3

💡 Hint

Look for the official syntax for version policy in TensorFlow Serving config.

The 'model_version_policy' with 'latest' and 'num_versions' is the correct way to specify how many versions to keep.

🔧 Debug

expert

Why does this TensorFlow Serving gRPC request fail with UNAVAILABLE error?

You run this Python gRPC client code to request predictions but get an UNAVAILABLE error. What is the most likely cause?

TensorFlow

import grpc
import tensorflow as tf
from tensorflow_serving.apis import prediction_service_pb2_grpc, predict_pb2

channel = grpc.insecure_channel('localhost:8500')
stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = 'my_model'
request.model_spec.signature_name = 'serving_default'
request.inputs['input_1'].CopyFrom(tf.make_tensor_proto([[1.0, 2.0, 3.0]]))
response = stub.Predict(request, timeout=5.0)
print(response)

A The input tensor name 'input_1' does not match the model signature

B TensorFlow Serving is not running or not listening on port 8501

C The gRPC channel is insecure and requires SSL credentials

D The model name 'my_model' is misspelled in the request

💡 Hint

UNAVAILABLE usually means the client cannot connect to the server.

The UNAVAILABLE error indicates the server is unreachable, often because TensorFlow Serving is not running or the port is incorrect.

Practice

1. What is the main purpose of TensorFlow Serving in machine learning?

easy

A. To clean and preprocess data before training

B. To train new machine learning models

C. To visualize model training progress

D. To deploy trained models for real-time predictions

Solution

1. Step 1: Understand TensorFlow Serving's role

   TensorFlow Serving is designed to deploy models so they can be used to make predictions quickly.

2. Step 2: Differentiate from training and preprocessing

   Training and preprocessing happen before serving; TensorFlow Serving focuses on using the trained model.

3. Final Answer:

   To deploy trained models for real-time predictions -> Option D

4. Quick Check:

   Serving = Deploy models for predictions [OK]

Hint: Serving means deploying models to answer queries fast [OK]

Common Mistakes:

• Confusing serving with training
• Thinking it preprocesses data
• Assuming it visualizes training

2. Which command correctly starts TensorFlow Serving to serve a model saved at /models/my_model?

easy

A. tensorflow_model_server --model_base_path=/models/my_model --rest_api_port=8501

B. tensorflow_serve --model_path=/models/my_model --port=8501

C. serve_tf_model --model_dir=/models/my_model --grpc_port=8501

D. start_tf_server --model=/models/my_model --api_port=8501

Solution

1. Step 1: Identify the official TensorFlow Serving command

   The official command is tensorflow_model_server with flags like --model_base_path and --rest_api_port.

2. Step 2: Match the correct flags and path

   tensorflow_model_server --model_base_path=/models/my_model --rest_api_port=8501 uses correct command and flags pointing to /models/my_model and port 8501.

3. Final Answer:

   tensorflow_model_server --model_base_path=/models/my_model --rest_api_port=8501 -> Option A

4. Quick Check:

   Correct command and flags = tensorflow_model_server --model_base_path=/models/my_model --rest_api_port=8501 [OK]

Hint: Look for 'tensorflow_model_server' and '--model_base_path' flags [OK]

Common Mistakes:

• Using wrong command name
• Incorrect flag names
• Wrong model path flag

3. Given this Python code snippet sending data to TensorFlow Serving REST API, what is the expected output if the model predicts class 1 with 0.85 confidence?

import requests
import json

url = 'http://localhost:8501/v1/models/my_model:predict'
data = {"instances": [[5.1, 3.5, 1.4, 0.2]]}
response = requests.post(url, json=data)
print(response.json())

medium

A. {"predictions": [[1, 0.85]]}

B. {"predictions": [[1]]}

C. {"predictions": [[0.85, 1]]}

D. {"error": "Model not found"}

Solution

1. Step 1: Understand the REST API response format

   TensorFlow Serving returns predictions as a list of outputs matching the model's output shape. For classification, usually a class label or probabilities.

2. Step 2: Match the expected output for class prediction

   If the model predicts class 1 with confidence 0.85, the output is typically the class label only, e.g., [1].

3. Final Answer:

   {"predictions": [[1]]} -> Option B

4. Quick Check:

   Prediction output = class label list [OK]

Hint: REST API returns predictions list matching model output shape [OK]

Common Mistakes:

• Confusing confidence with class label
• Expecting confidence and label together
• Ignoring model output format

4. You started TensorFlow Serving with this command:
tensorflow_model_server --model_base_path=/models/my_model --rest_api_port=8501
But when you send a request, you get a 404 error. What is the most likely cause?

medium

A. The model folder path is incorrect or missing the version subfolder

B. The REST API port 8501 is blocked by firewall

C. The client request JSON format is wrong

D. TensorFlow Serving does not support REST API

Solution

1. Step 1: Check model folder structure requirement

   TensorFlow Serving expects the model folder to contain version subfolders like 1/, 2/ with saved model files inside.

2. Step 2: Understand 404 error meaning

   A 404 error means the server cannot find the requested model, often due to missing or wrong model path or version folder.

3. Final Answer:

   The model folder path is incorrect or missing the version subfolder -> Option A

4. Quick Check:

   404 error = model path/version issue [OK]

Hint: Model folder must have version subfolders like '1/' [OK]

Common Mistakes:

• Ignoring model folder versioning
• Assuming REST API unsupported
• Blaming firewall without checking model path

5. You want to serve two different versions of a model simultaneously using TensorFlow Serving. Which setup allows clients to request a specific version?

hard

A. Rename model folders to different names and serve them under separate model names

B. Run two separate TensorFlow Serving servers on different ports, each serving one version

C. Place version folders (e.g., 1/, 2/) inside the model base path and use the version parameter in the request URL

D. TensorFlow Serving does not support serving multiple versions simultaneously

Solution

1. Step 1: Understand versioning in TensorFlow Serving

   TensorFlow Serving supports multiple versions by placing them as subfolders inside the model base path, e.g., 1/, 2/.

2. Step 2: How clients request specific versions

   Clients can specify the version in the request URL to get predictions from a particular version.

3. Final Answer:

   Place version folders (e.g., 1/, 2/) inside the model base path and use the version parameter in the request URL -> Option C

4. Quick Check:

   Version folders + version param = multiple versions [OK]

Hint: Use version folders and version param in request URL [OK]

Common Mistakes:

• Thinking separate servers are needed
• Renaming folders instead of versioning
• Believing multiple versions unsupported