Flask API for model serving in ML Python - Model Metrics & Evaluation

Metrics & Evaluation - Flask API for model serving

Which metric matters for Flask API model serving and WHY

When serving a model with a Flask API, the key metrics to watch are latency and accuracy. Latency tells us how fast the API responds to a request, which affects user experience. Accuracy tells us if the model predictions are correct. Both matter because a slow or wrong prediction hurts users.

Confusion matrix example for classification model served by Flask API

      | Predicted Positive | Predicted Negative |
      |--------------------|--------------------|
      | True Positive (TP): 50 | False Negative (FN): 10 |
      | False Positive (FP): 5 | True Negative (TN): 35 |

      Total samples = 50 + 10 + 5 + 35 = 100
    

From this, we calculate:

• Precision = TP / (TP + FP) = 50 / (50 + 5) = 0.91
• Recall = TP / (TP + FN) = 50 / (50 + 10) = 0.83
• F1 Score = 2 * (Precision * Recall) / (Precision + Recall) ≈ 0.87

Precision vs Recall tradeoff with Flask API example

Imagine your Flask API serves a spam detection model:

• High Precision: The API rarely marks good emails as spam. Users trust it because false alarms are low.
• High Recall: The API catches almost all spam emails, but may mark some good emails as spam.

Depending on user needs, you might prefer one over the other. For example, if missing spam is worse, prioritize recall. If annoying users with false spam is worse, prioritize precision.

Good vs Bad metric values for Flask API model serving

Good metrics:

• Accuracy above 85% for balanced data
• Precision and recall both above 80%
• API latency under 200 milliseconds

Bad metrics:

• Accuracy near random guess (e.g., 50% for binary)
• Precision very low (e.g., 30%) causing many false positives
• Recall very low (e.g., 20%) missing many true cases
• API latency over 1 second causing slow user experience

Common pitfalls in Flask API model serving metrics

• Accuracy paradox: High accuracy but poor recall or precision due to imbalanced data.
• Data leakage: Model trained on data similar to test data inflates metrics but fails in real API use.
• Overfitting: Model performs well on training but poorly on API requests from new data.
• Ignoring latency: A model with great accuracy but slow API response frustrates users.

Self-check question

Your Flask API model has 98% accuracy but only 12% recall on fraud detection. Is it good for production? Why or why not?

Answer: No, it is not good. The low recall means the model misses most fraud cases, which is dangerous. Even with high accuracy, missing fraud is costly. You should improve recall before production.

Key Result

For Flask API model serving, balance accuracy and latency, and carefully monitor precision and recall to ensure useful, timely predictions.

Practice

1. What is the main purpose of using Flask in model serving for machine learning?

easy

A. To train machine learning models faster

B. To create a web API that allows clients to send data and get predictions

C. To visualize data in charts and graphs

D. To store large datasets efficiently

Solution

1. Step 1: Understand Flask's role

   Flask is a lightweight web framework used to build web applications and APIs.

2. Step 2: Connect Flask to model serving

   In machine learning, Flask is used to create an API endpoint where clients can send input data and receive model predictions.

3. Final Answer:

   To create a web API that allows clients to send data and get predictions -> Option B

4. Quick Check:

   Flask for API = Create prediction endpoint [OK]

Hint: Flask makes your model accessible via web requests [OK]

Common Mistakes:

• Thinking Flask trains models
• Confusing Flask with data storage tools
• Assuming Flask is for data visualization

2. Which of the following is the correct way to define a Flask route for a POST request to serve a model prediction?

easy

A. @app.route('/predict', methods=['GET'])

B. @app.route('/predict')

C. @app.route('/predict', methods=['POST'])

D. @app.route('/predict', method='POST')

Solution

1. Step 1: Recall Flask route syntax

   Flask routes use @app.route decorator with a path and a list of HTTP methods.

2. Step 2: Identify correct method for sending data

   POST method is used to send data to the server for processing, suitable for model prediction.

3. Final Answer:

   @app.route('/predict', methods=['POST']) -> Option C

4. Quick Check:

   POST method for data sending = Correct route [OK]

Hint: Use methods=['POST'] to accept data in Flask route [OK]

Common Mistakes:

• Using GET instead of POST for sending data
• Missing 'methods' keyword or using wrong syntax
• Assuming default route accepts POST

3. Given the Flask code snippet below, what will be the output when a POST request with JSON {"input": [1, 2, 3]} is sent?

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    result = sum(data['input'])
    return jsonify({'prediction': result})

medium

A. {"prediction": 6}

B. {"prediction": [1, 2, 3]}

C. {"prediction": 123}

D. Error: KeyError

Solution

1. Step 1: Extract input data from JSON

   The code uses request.get_json() to get {'input': [1, 2, 3]} and accesses data['input'] which is [1, 2, 3].

2. Step 2: Calculate sum of input list

   sum([1, 2, 3]) equals 6, which is stored in result and returned as JSON.

3. Final Answer:

   {"prediction": 6} -> Option A

4. Quick Check:

   Sum of [1,2,3] = 6 [OK]

Hint: Sum list input to get prediction value [OK]

Common Mistakes:

• Returning the list instead of sum
• Concatenating numbers as string
• Missing JSON key causing KeyError

4. Identify the error in the following Flask API code for model serving and select the fix:

from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json()
    prediction = model.predict(data['input'])
    return jsonify({'prediction': prediction.tolist()})

medium

A. Change request.json() to request.get_json()

B. Change methods=['POST'] to methods=['GET']

C. Remove .tolist() from prediction

D. Add @app.run() after function

Solution

1. Step 1: Check how JSON data is accessed

   request.json() is incorrect; Flask uses request.get_json() to parse JSON body.

2. Step 2: Confirm other parts are correct

   methods=['POST'] is correct for sending data; .tolist() is valid if prediction is numpy array; @app.run() is not a decorator.

3. Final Answer:

   Change request.json() to request.get_json() -> Option A

4. Quick Check:

   Use request.get_json() to parse JSON [OK]

Hint: Use request.get_json() to read JSON body [OK]

Common Mistakes:

• Using request.json() instead of get_json()
• Confusing HTTP methods for data sending
• Misusing Flask decorators

5. You want to serve a scikit-learn model using Flask. The model expects input features as a 2D list. Which approach correctly prepares the input inside the Flask route before prediction?

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    # Prepare input for model
    input_features = ???
    prediction = model.predict(input_features)
    return jsonify({'prediction': prediction.tolist()})

hard

A. input_features = data['features'].tolist() # Convert to list

B. input_features = data['features'] # Use list directly

C. input_features = data['features'][0] # Use first element only

D. input_features = [data['features']] # Wrap list inside another list

Solution

1. Step 1: Understand model input shape

   Scikit-learn models expect 2D arrays: shape (n_samples, n_features). A single sample must be wrapped in an outer list.

2. Step 2: Prepare input accordingly

   Wrapping data['features'] (a list of features) inside another list creates a 2D list suitable for prediction.

3. Final Answer:

   input_features = [data['features']] # Wrap list inside another list -> Option D

4. Quick Check:

   Wrap features in list for 2D input [OK]

Hint: Wrap single feature list in another list for 2D input [OK]

Common Mistakes:

• Passing 1D list directly causing shape errors
• Using only first element losing features
• Calling .tolist() on a list (already a list)