API-based deployment in Prompt Engineering / GenAI - ML Experiment: Train & Evaluate

Experiment - API-based deployment

Problem:You have trained a text classification model that works well locally. Now you want to deploy it as an API so others can send text and get predictions.

Current Metrics:Model accuracy on test data: 90%. No deployment yet.

Issue:The model is not accessible remotely. Users cannot get predictions without running code locally.

Your Task

Deploy the trained text classification model as a REST API that accepts text input and returns predicted labels. The API should respond within 1 second for a single request.

Use a lightweight web framework (e.g., FastAPI or Flask).

Do not change the model architecture or retrain the model.

Ensure the API handles JSON input and output.

Hint 1

Hint 2

Hint 3

Hint 4

Solution

Prompt Engineering / GenAI

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import uvicorn
import joblib

# Define input data model
class TextInput(BaseModel):
    text: str

# Load the trained model once
model = joblib.load('text_classifier.joblib')

app = FastAPI()

@app.post('/predict')
async def predict(input: TextInput):
    text = input.text
    if not text:
        raise HTTPException(status_code=400, detail='Text input is empty')
    # Model expects a list of texts
    prediction = model.predict([text])[0]
    return {'label': prediction}

if __name__ == '__main__':
    uvicorn.run(app, host='0.0.0.0', port=8000)

Added FastAPI web server to serve the model as an API.

Created a POST /predict endpoint that accepts JSON input with a 'text' field.

Loaded the trained model once at startup to improve response time.

Returned prediction label as JSON output.

Results Interpretation

Before deployment: Model accuracy 90%, no remote access.

After deployment: Model accuracy unchanged at 90%, API responds within 200ms, enabling remote predictions.

Deploying a trained model as an API makes it accessible to others without changing the model. Loading the model once and using a lightweight web framework ensures fast responses.

Bonus Experiment

Add input validation to reject empty or very long texts and return meaningful error messages.

💡 Hint

Use Pydantic validators or FastAPI request validation features to check input length and content before prediction.