REST API for predictions in TensorFlow

Introduction

A REST API lets you send data to your machine learning model from anywhere and get predictions back easily.

You want to let a website ask your model for predictions.

You want a mobile app to get predictions from your model.

You want to share your model with others without giving them the code.

You want to automate predictions from other programs or services.

Syntax

TensorFlow

from flask import Flask, request, jsonify
import tensorflow as tf

app = Flask(__name__)
model = tf.keras.models.load_model('your_model_path')

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json['input']
    data_np = tf.convert_to_tensor([data])
    prediction = model.predict(data_np)
    return jsonify({'prediction': prediction.tolist()})

if __name__ == '__main__':
    app.run(debug=True)

Use POST method to send data safely in the request body.

The input data should be in JSON format for easy parsing.

Examples

Send a JSON array of features to the API using curl command.

TensorFlow

curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"input": [5.1, 3.5, 1.4, 0.2]}'

Use Python requests library to send data and get prediction.

TensorFlow

import requests
response = requests.post('http://localhost:5000/predict', json={'input': [5.1, 3.5, 1.4, 0.2]})
print(response.json())

Sample Model

This code trains a simple model on dummy data and creates a Flask REST API to predict the class for new input data sent as JSON.

TensorFlow

from flask import Flask, request, jsonify
import tensorflow as tf
import numpy as np

# Create a simple model for demonstration
model = tf.keras.Sequential([
    tf.keras.layers.Dense(3, activation='softmax', input_shape=(4,))
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# Dummy training data
X_train = np.array([[5.1, 3.5, 1.4, 0.2],
                    [6.2, 3.4, 5.4, 2.3],
                    [5.9, 3.0, 5.1, 1.8]])
y_train = np.array([0, 2, 2])

model.fit(X_train, y_train, epochs=5, verbose=0)

app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json['input']
    data_np = np.array([data])
    prediction = model.predict(data_np)
    predicted_class = int(np.argmax(prediction[0]))
    return jsonify({'prediction': predicted_class})

if __name__ == '__main__':
    app.run(debug=False)

Important Notes

Make sure your input data shape matches what the model expects.

Use a production-ready server like Gunicorn for real deployment.

Always validate and sanitize input data to avoid errors.

Summary

A REST API lets you send data and get predictions from your ML model easily.

Use Flask to create a simple API endpoint that loads your TensorFlow model.

Send input data as JSON and return predictions as JSON for easy use.

Practice

1. What is the main purpose of creating a REST API for a TensorFlow model?

easy

A. To save the model in a different file format

B. To train the model faster on a server

C. To visualize the model's architecture

D. To allow other programs to send data and get predictions easily

Solution

1. Step 1: Understand REST API role

   A REST API acts as a bridge to send data and receive results from a model.

2. Step 2: Match purpose with options

   Only To allow other programs to send data and get predictions easily describes this role correctly as it enables easy data exchange for predictions.

3. Final Answer:

   To allow other programs to send data and get predictions easily -> Option D

4. Quick Check:

   REST API purpose = Easy data exchange for predictions [OK]

Hint: REST API = send input, get output from model [OK]

Common Mistakes:

• Confusing API with training process
• Thinking API is for visualization
• Assuming API changes model files

2. Which Flask code snippet correctly defines a POST endpoint to receive JSON input for predictions?

easy

A. app.post('/predict')\ndef predict():\n data = request.get_json()

B. app.route('/predict')\ndef predict():\n data = request.get_json()

C. app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json()

D. app.get('/predict')\ndef predict():\n data = request.get_json()

Solution

1. Step 1: Identify correct Flask route syntax for POST

   Flask uses @app.route with methods=['POST'] to define POST endpoints.

2. Step 2: Check code snippets

   Only app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json() uses @app.route with methods=['POST'] and calls request.get_json() correctly.

3. Final Answer:

   app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json() -> Option C

4. Quick Check:

   Flask POST route = @app.route with methods=['POST'] [OK]

Hint: Flask POST = @app.route with methods=['POST'] [OK]

Common Mistakes:

• Using app.post decorator (not standard Flask)
• Missing methods=['POST'] in route
• Using app.get for POST requests

3. Given this Flask code snippet, what will be the output if the input JSON is {'input': [1, 2, 3]} and the model predicts sum of inputs?

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    inputs = data['input']
    prediction = sum(inputs)
    return jsonify({'prediction': prediction})

medium

A. {"prediction": 6}

B. {"prediction": [1, 2, 3]}

C. {"prediction": 123}

D. Error: KeyError

Solution

1. Step 1: Extract input and compute prediction

   The input JSON has 'input': [1, 2, 3]. The code sums these values: 1+2+3=6.

2. Step 2: Return prediction as JSON

   The code returns jsonify({'prediction': 6}), so output is {"prediction": 6}.

3. Final Answer:

   {"prediction": 6} -> Option A

4. Quick Check:

   Sum([1,2,3]) = 6 [OK]

Hint: Sum input list to get prediction output [OK]

Common Mistakes:

• Returning input list instead of sum
• Concatenating numbers as string
• Misreading JSON keys causing KeyError

4. You wrote this Flask API code but get an error when sending POST requests. What is the likely cause?

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict')
def predict():
    data = request.get_json()
    prediction = model.predict(data['input'])
    return jsonify({'prediction': prediction.tolist()})

medium

A. model.predict() cannot accept JSON data

B. Missing methods=['POST'] in @app.route decorator

C. request.get_json() only works with GET requests

D. jsonify() cannot convert lists

Solution

1. Step 1: Check route decorator for HTTP methods

   The route lacks methods=['POST'], so it defaults to GET only, causing errors on POST requests.

2. Step 2: Confirm other parts are correct

   model.predict() can accept arrays, get_json() works with POST, and jsonify converts lists fine.

3. Final Answer:

   Missing methods=['POST'] in @app.route decorator -> Option B

4. Quick Check:

   POST route needs methods=['POST'] [OK]

Hint: Add methods=['POST'] to route for POST requests [OK]

Common Mistakes:

• Forgetting to specify POST method in route
• Assuming get_json works only with GET
• Thinking jsonify can't handle lists

5. You want to deploy a TensorFlow model with a REST API that accepts multiple inputs and returns predictions for each. Which approach is best to handle batch inputs in your Flask API?

hard

A. Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions

B. Accept one input at a time, call model.predict repeatedly for each

C. Save each input to a file, then load and predict later

D. Use GET requests with inputs in URL parameters

Solution

1. Step 1: Understand batch prediction needs

   Batch prediction means processing many inputs at once for efficiency.

2. Step 2: Choose method to handle batch inputs

   Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions accepts JSON list, converts to array, predicts all at once, then returns all predictions, which is efficient and standard.

3. Final Answer:

   Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions -> Option A

4. Quick Check:

   Batch input = JSON list + batch predict [OK]

Hint: Batch predict by sending list and returning list [OK]

Common Mistakes:

• Predicting one input at a time (slow)
• Using files instead of direct JSON
• Using GET for large input data