REST API for predictions in TensorFlow - Model Pipeline Trace

Model Pipeline - REST API for predictions

This pipeline shows how a machine learning model is trained and then used in a REST API to make predictions when new data arrives. The API receives data, processes it, runs the model, and returns the prediction.

Data Flow - 5 Stages

1Raw data input

1000 rows x 10 columns→Collect raw features from dataset→1000 rows x 10 columns

[[5.1, 3.5, 1.4, ..., 0.2], ...]

↓

2Preprocessing

1000 rows x 10 columns→Normalize features to range 0-1→1000 rows x 10 columns

[[0.52, 0.7, 0.14, ..., 0.02], ...]

↓

3Train/test split

1000 rows x 10 columns→Split data into 800 training and 200 testing rows→Train: 800 rows x 10 columns, Test: 200 rows x 10 columns

Train sample: [0.52, 0.7, 0.14, ..., 0.02]

↓

4Model training

800 rows x 10 columns→Train neural network classifier→Trained model with input shape 10

Model weights updated after each epoch

↓

5Model deployment

New data sample 1 row x 10 columns→Serve model via REST API→Prediction output (class probabilities)

[0.1, 0.7, 0.2]

Training Trace - Epoch by Epoch

Loss
0.7 | *       
0.6 |  *      
0.5 |   *     
0.4 |    *    
0.3 |     *   
0.2 |      *  
    +---------
     1 2 3 4 5
     Epochs

Epoch	Loss ↓	Accuracy ↑	Observation
1	0.65	0.60	Model starts learning, loss high, accuracy low
2	0.48	0.75	Loss decreases, accuracy improves
3	0.35	0.85	Model converging, better predictions
4	0.28	0.90	Loss low, accuracy high
5	0.22	0.93	Training stabilizes with good accuracy

Prediction Trace - 4 Layers

Layer 1: API receives JSON input

Layer 2: Preprocessing inside API

Layer 3: Model prediction

Layer 4: API response

Model Quiz - 3 Questions

What happens to the loss value during training epochs?

A It increases steadily

B It decreases steadily

C It stays the same

D It randomly jumps up and down

Key Insight

This visualization shows how a trained model can be wrapped inside a REST API to receive new data, process it, and return predictions quickly. The training trace confirms the model learns well, and the prediction trace shows the smooth flow from input to output in the API.

Practice

1. What is the main purpose of creating a REST API for a TensorFlow model?

easy

A. To save the model in a different file format

B. To train the model faster on a server

C. To visualize the model's architecture

D. To allow other programs to send data and get predictions easily

Solution

1. Step 1: Understand REST API role

   A REST API acts as a bridge to send data and receive results from a model.

2. Step 2: Match purpose with options

   Only To allow other programs to send data and get predictions easily describes this role correctly as it enables easy data exchange for predictions.

3. Final Answer:

   To allow other programs to send data and get predictions easily -> Option D

4. Quick Check:

   REST API purpose = Easy data exchange for predictions [OK]

Hint: REST API = send input, get output from model [OK]

Common Mistakes:

• Confusing API with training process
• Thinking API is for visualization
• Assuming API changes model files

2. Which Flask code snippet correctly defines a POST endpoint to receive JSON input for predictions?

easy

A. app.post('/predict')\ndef predict():\n data = request.get_json()

B. app.route('/predict')\ndef predict():\n data = request.get_json()

C. app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json()

D. app.get('/predict')\ndef predict():\n data = request.get_json()

Solution

1. Step 1: Identify correct Flask route syntax for POST

   Flask uses @app.route with methods=['POST'] to define POST endpoints.

2. Step 2: Check code snippets

   Only app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json() uses @app.route with methods=['POST'] and calls request.get_json() correctly.

3. Final Answer:

   app.route('/predict', methods=['POST'])\ndef predict():\n data = request.get_json() -> Option C

4. Quick Check:

   Flask POST route = @app.route with methods=['POST'] [OK]

Hint: Flask POST = @app.route with methods=['POST'] [OK]

Common Mistakes:

• Using app.post decorator (not standard Flask)
• Missing methods=['POST'] in route
• Using app.get for POST requests

3. Given this Flask code snippet, what will be the output if the input JSON is {'input': [1, 2, 3]} and the model predicts sum of inputs?

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    inputs = data['input']
    prediction = sum(inputs)
    return jsonify({'prediction': prediction})

medium

A. {"prediction": 6}

B. {"prediction": [1, 2, 3]}

C. {"prediction": 123}

D. Error: KeyError

Solution

1. Step 1: Extract input and compute prediction

   The input JSON has 'input': [1, 2, 3]. The code sums these values: 1+2+3=6.

2. Step 2: Return prediction as JSON

   The code returns jsonify({'prediction': 6}), so output is {"prediction": 6}.

3. Final Answer:

   {"prediction": 6} -> Option A

4. Quick Check:

   Sum([1,2,3]) = 6 [OK]

Hint: Sum input list to get prediction output [OK]

Common Mistakes:

• Returning input list instead of sum
• Concatenating numbers as string
• Misreading JSON keys causing KeyError

4. You wrote this Flask API code but get an error when sending POST requests. What is the likely cause?

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict')
def predict():
    data = request.get_json()
    prediction = model.predict(data['input'])
    return jsonify({'prediction': prediction.tolist()})

medium

A. model.predict() cannot accept JSON data

B. Missing methods=['POST'] in @app.route decorator

C. request.get_json() only works with GET requests

D. jsonify() cannot convert lists

Solution

1. Step 1: Check route decorator for HTTP methods

   The route lacks methods=['POST'], so it defaults to GET only, causing errors on POST requests.

2. Step 2: Confirm other parts are correct

   model.predict() can accept arrays, get_json() works with POST, and jsonify converts lists fine.

3. Final Answer:

   Missing methods=['POST'] in @app.route decorator -> Option B

4. Quick Check:

   POST route needs methods=['POST'] [OK]

Hint: Add methods=['POST'] to route for POST requests [OK]

Common Mistakes:

• Forgetting to specify POST method in route
• Assuming get_json works only with GET
• Thinking jsonify can't handle lists

5. You want to deploy a TensorFlow model with a REST API that accepts multiple inputs and returns predictions for each. Which approach is best to handle batch inputs in your Flask API?

hard

A. Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions

B. Accept one input at a time, call model.predict repeatedly for each

C. Save each input to a file, then load and predict later

D. Use GET requests with inputs in URL parameters

Solution

1. Step 1: Understand batch prediction needs

   Batch prediction means processing many inputs at once for efficiency.

2. Step 2: Choose method to handle batch inputs

   Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions accepts JSON list, converts to array, predicts all at once, then returns all predictions, which is efficient and standard.

3. Final Answer:

   Accept a JSON list of inputs, convert to numpy array, predict batch, return list of predictions -> Option A

4. Quick Check:

   Batch input = JSON list + batch predict [OK]

Hint: Batch predict by sending list and returning list [OK]

Common Mistakes:

• Predicting one input at a time (slow)
• Using files instead of direct JSON
• Using GET for large input data