LangServe for API deployment in LangChain - Step-by-Step Execution

Concept Flow - LangServe for API deployment

Define LangChain Model

↓

Create FastAPI App

↓

Add Routes with LangServe

↓

Start API Server

↓

Receive API Request

↓

Process Request with Model

↓

Send Response Back

This flow shows how you define a model, create a FastAPI app, add LangServe routes for the model, start the server, and handle API requests.

Execution Sample

LangChain

from fastapi import FastAPI
from langserve import add_routes
from langchain.chat_models import ChatOpenAI
import uvicorn

model = ChatOpenAI()
app = FastAPI()
add_routes(app, model, path="/chat")
uvicorn.run(app, host="localhost", port=8000)

This code sets up a ChatOpenAI model, creates a FastAPI app, adds LangServe routes, and starts the API server with uvicorn.

Execution Table

Step	Action	State Change	Result
1	Import FastAPI, add_routes, ChatOpenAI, uvicorn	Modules ready	No output
2	Create ChatOpenAI model instance	model variable set	Model ready to use
3	Create FastAPI app instance	app variable set	App ready to add routes
4	add_routes(app, model, path="/chat")	app now has routes	App can serve model via API
5	Start API server with uvicorn.run(app)	Server running	API endpoint available at http://localhost:8000/chat/invoke
6	Receive API request	Request received	Ready to process
7	Process request using model	Model generates response	Response ready
8	Send response back to client	Response sent	Client receives answer
9	No more requests or server stopped	Server stops	API unavailable

💡 Server stops when manually stopped or no requests remain

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
model	None	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance
app	None	None	FastAPI instance	FastAPI with LangServe routes	Server running	Server stopped or running

Key Moments - 3 Insights

Why do we need to call add_routes(app, model) before serving?

Because it registers the model's invoke/call methods as API endpoints on the FastAPI app, as shown in step 4 of the execution_table.

What happens if we call uvicorn.run(app) without adding routes?

The server starts but has no endpoints for the model, so requests to /chat won't work (refer to step 5 and 7).

How does the API server handle multiple requests?

Each request triggers step 6 and 7 repeatedly, processing with the model via the registered routes and sending responses, as shown in the loop between steps 6 to 8.

Visual Quiz - 3 Questions

Look at the execution_table, what is the state of 'app' after step 4?

A FastAPI with LangServe routes

B FastAPI instance without routes

C None

D Server stopped

💡 Hint

Check the 'State Change' column at step 4 in execution_table

At which step does the API server start listening for requests?

A Step 7

B Step 3

C Step 5

D Step 2

💡 Hint

Look for 'Server running' in the 'State Change' column

If you skip add_routes(app, model), what will happen when a request arrives at /chat?

A Request is processed normally

B 404 error or no response generated

C Server crashes immediately

D Server refuses to start

💡 Hint

Refer to key_moments about missing routes and step 7 in execution_table

Concept Snapshot

LangServe API Deployment:
1. Import FastAPI, langserve.add_routes, langchain.chat_models.ChatOpenAI, uvicorn.
2. Create model instance.
3. Create FastAPI app.
4. add_routes(app, model, path="/chat").
5. Start server with uvicorn.run(app).
6. Server handles requests by passing them to model.
7. Responses sent back to clients.

Full Transcript

This visual execution shows how to deploy an API using LangServe with LangChain models. First, import FastAPI, add_routes from langserve, ChatOpenAI, and uvicorn. Create a ChatOpenAI model instance. Create a FastAPI app. Add routes with add_routes(app, model, path="/chat"). Start the server with uvicorn.run(app). When a request comes to /chat/invoke, the server passes it to the model, which processes it and returns a response. The server sends this back to the client. This repeats for each request until stopped.