TensorFlow Serving setup - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the primary purpose of TensorFlow Serving?

TensorFlow Serving is a system designed to:

A Visualize model training metrics and graphs

B Train machine learning models efficiently on large datasets

C Serve machine learning models in production environments with high performance

D Convert models from TensorFlow format to other frameworks

💡 Hint

Think about what happens after a model is trained and needs to be used by applications.

TensorFlow Serving is built to deploy and serve trained models so applications can use them for predictions quickly and reliably.

❓ Predict Output

intermediate

What output does this TensorFlow Serving command produce?

Consider the command below to start TensorFlow Serving with a model named my_model located at /models/my_model/1/:

tensorflow_model_server --rest_api_port=8501 --model_name=my_model --model_base_path=/models/my_model/1/

What will happen after running this command?

A TensorFlow Serving starts and listens on REST port 8501 serving the model 'my_model'

B TensorFlow Serving starts but fails because the model path must not include the version number

C TensorFlow Serving starts but listens on gRPC port 8501 instead of REST

D TensorFlow Serving fails to start because the model name is missing

💡 Hint

Check the meaning of --rest_api_port and the model path format.

The command starts TensorFlow Serving with REST API on port 8501 serving the specified model version at the given path.

❓ Hyperparameter

advanced

Which TensorFlow Serving flag controls the number of model versions kept in memory?

When serving models, you want to limit memory usage by keeping only the latest versions loaded. Which flag controls this behavior?

A --model_version_policy

B --max_num_load_versions

C --file_system_poll_wait_seconds

D --max_num_load_retries

💡 Hint

Look for a flag related to version management policy.

The --model_version_policy flag lets you specify which versions to keep loaded, such as the latest N versions.

🔧 Debug

advanced

Why does this TensorFlow Serving startup command fail?

Given the command below:

tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model

The server fails to start and shows an error about missing model version. What is the likely cause?

A The model_name must not contain underscores

B TensorFlow Serving requires a config file instead of command line flags

C The command is missing the --rest_api_port flag

D The model_base_path must include a version subdirectory like /models/my_model/1/

💡 Hint

Check the expected folder structure for model versions.

TensorFlow Serving expects the model_base_path to point to a directory containing version subdirectories (e.g., 1, 2). Without this, it cannot find a valid model version.

❓ Metrics

expert

Which metric indicates TensorFlow Serving is successfully processing prediction requests?

After deploying a model with TensorFlow Serving, you want to monitor if it is handling prediction requests correctly. Which metric should you check?

A tensorflow_serving_model_load_seconds_count

B tensorflow_serving_predict_requests_total

C tensorflow_serving_model_version_status

D tensorflow_serving_num_load_failures

💡 Hint

Look for a metric counting prediction requests.

The metric tensorflow_serving_predict_requests_total counts the total number of prediction requests processed, indicating active serving.

Practice

1. What is the main purpose of TensorFlow Serving in a machine learning workflow?

easy

A. To visualize training data and model metrics

B. To train new machine learning models from scratch

C. To preprocess data before training

D. To deploy trained models for real-time predictions

Solution

1. Step 1: Understand TensorFlow Serving's role

   TensorFlow Serving is designed to make trained models available for real-time use without retraining.

2. Step 2: Differentiate from other tasks

   Training, visualization, and preprocessing are separate steps not handled by TensorFlow Serving.

3. Final Answer:

   To deploy trained models for real-time predictions -> Option D

4. Quick Check:

   Serving = deploy models [OK]

Hint: Serving means making models ready for prediction [OK]

Common Mistakes:

• Confusing serving with training
• Thinking it handles data preprocessing
• Assuming it visualizes metrics

2. Which command correctly starts TensorFlow Serving to serve a model saved in the directory /models/my_model/1/ with model name my_model?

easy

A. tensorflow_model_server --model_name my_model --model_base_path /models/my_model/1/

B. tensorflow_model_server --model_name=my_model --model_path=/models/my_model/1/

C. tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model/1/

D. tensorflow_serve --model_name=my_model --model_base_path=/models/my_model/1/

Solution

1. Step 1: Identify correct flags for TensorFlow Serving

   The correct flags are --model_name and --model_base_path with equal signs.

2. Step 2: Check command correctness

   tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model/1/ uses correct command and flags with equal signs; others have syntax errors or wrong command name.

3. Final Answer:

   tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model/1/ -> Option C

4. Quick Check:

   Correct flags and command = tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model/1/ [OK]

Hint: Use equal signs with flags and correct command name [OK]

Common Mistakes:

• Using spaces instead of equal signs in flags
• Wrong command name like tensorflow_serve
• Using incorrect flag names like --model_path

3. Given the following Python code snippet to send data to a TensorFlow Serving REST API, what will be the printed output if the server returns a prediction of 0.85?

import requests
import json

url = 'http://localhost:8501/v1/models/my_model:predict'
data = {"instances": [[1.0, 2.0, 3.0]]}
response = requests.post(url, json=data)
prediction = response.json()['predictions'][0]
print(prediction)

medium

A. 0.85

B. [0.85]

C. {'predictions': [0.85]}

D. Error: KeyError

Solution

1. Step 1: Understand the response JSON structure

   The server returns a JSON with key 'predictions' holding a list of prediction values, e.g., {'predictions': [0.85]}.

2. Step 2: Extract the first prediction value

   Accessing response.json()['predictions'][0] extracts the float 0.85.

3. Final Answer:

   0.85 -> Option A

4. Quick Check:

   Extract first prediction value = 0.85 [OK]

Hint: Access ['predictions'][0] to get single prediction value [OK]

Common Mistakes:

• Printing the whole JSON instead of the value
• Forgetting to index the first prediction
• Assuming prediction is a list, not a float

4. You tried to start TensorFlow Serving with the command tensorflow_model_server --model_name=my_model --model_base_path=/models/my_model/1/ but got an error saying the model path does not exist. What is the most likely cause?

medium

A. The command should use --model_path instead of --model_base_path

B. The model directory path is incorrect or missing the version subfolder

C. TensorFlow Serving requires the model to be in a .pb file, not a folder

D. The model name must be the full path, not just a name

Solution

1. Step 1: Check model directory structure requirements

   TensorFlow Serving expects model_base_path to point to a directory containing version subfolders like '1', '2', etc.

2. Step 2: Identify common path mistakes

   If the path misses the version folder or is incorrect, the server cannot find the model and throws an error.

3. Final Answer:

   The model directory path is incorrect or missing the version subfolder -> Option B

4. Quick Check:

   Model path must include version folder [OK]

Hint: Model path must include version folder like /1/ [OK]

Common Mistakes:

• Using --model_path instead of --model_base_path
• Pointing to a file instead of a directory
• Confusing model name with path

5. You want to update your TensorFlow model served by TensorFlow Serving without downtime. Which approach allows you to serve the new model version while keeping the old version available for requests?

hard

A. Add the new model version in a new subfolder and update the model config file

B. Stop TensorFlow Serving, replace the old model folder, then restart the server

C. Serve the new model on a different port and manually switch clients

D. Retrain the model inside TensorFlow Serving using the new data

Solution

1. Step 1: Understand TensorFlow Serving versioning

   TensorFlow Serving supports multiple model versions by placing each version in its own numbered subfolder.

2. Step 2: Use model config file to add new version

   Adding the new version folder and updating the config allows serving both versions simultaneously without downtime.

3. Final Answer:

   Add the new model version in a new subfolder and update the model config file -> Option A

4. Quick Check:

   Versioned folders + config update = zero downtime [OK]

Hint: Use versioned folders and config to update without downtime [OK]

Common Mistakes:

• Replacing model folder while server runs
• Restarting server causes downtime
• Trying to retrain model inside serving