LangServe for API deployment in LangChain - Step-by-Step Execution

Concept Flow - LangServe for API deployment

Define LangChain Model

↓

Create FastAPI App

↓

Add Routes with LangServe

↓

Start API Server

↓

Receive API Request

↓

Process Request with Model

↓

Send Response Back

This flow shows how you define a model, create a FastAPI app, add LangServe routes for the model, start the server, and handle API requests.

Execution Sample

LangChain

from fastapi import FastAPI
from langserve import add_routes
from langchain.chat_models import ChatOpenAI
import uvicorn

model = ChatOpenAI()
app = FastAPI()
add_routes(app, model, path="/chat")
uvicorn.run(app, host="localhost", port=8000)

This code sets up a ChatOpenAI model, creates a FastAPI app, adds LangServe routes, and starts the API server with uvicorn.

Execution Table

Step	Action	State Change	Result
1	Import FastAPI, add_routes, ChatOpenAI, uvicorn	Modules ready	No output
2	Create ChatOpenAI model instance	model variable set	Model ready to use
3	Create FastAPI app instance	app variable set	App ready to add routes
4	add_routes(app, model, path="/chat")	app now has routes	App can serve model via API
5	Start API server with uvicorn.run(app)	Server running	API endpoint available at http://localhost:8000/chat/invoke
6	Receive API request	Request received	Ready to process
7	Process request using model	Model generates response	Response ready
8	Send response back to client	Response sent	Client receives answer
9	No more requests or server stopped	Server stops	API unavailable

💡 Server stops when manually stopped or no requests remain

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
model	None	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance	ChatOpenAI instance
app	None	None	FastAPI instance	FastAPI with LangServe routes	Server running	Server stopped or running

Key Moments - 3 Insights

Why do we need to call add_routes(app, model) before serving?

Because it registers the model's invoke/call methods as API endpoints on the FastAPI app, as shown in step 4 of the execution_table.

What happens if we call uvicorn.run(app) without adding routes?

The server starts but has no endpoints for the model, so requests to /chat won't work (refer to step 5 and 7).

How does the API server handle multiple requests?

Each request triggers step 6 and 7 repeatedly, processing with the model via the registered routes and sending responses, as shown in the loop between steps 6 to 8.

Visual Quiz - 3 Questions

Look at the execution_table, what is the state of 'app' after step 4?

A FastAPI with LangServe routes

B FastAPI instance without routes

C None

D Server stopped

💡 Hint

Check the 'State Change' column at step 4 in execution_table

At which step does the API server start listening for requests?

A Step 7

B Step 3

C Step 5

D Step 2

💡 Hint

Look for 'Server running' in the 'State Change' column

If you skip add_routes(app, model), what will happen when a request arrives at /chat?

A Request is processed normally

B 404 error or no response generated

C Server crashes immediately

D Server refuses to start

💡 Hint

Refer to key_moments about missing routes and step 7 in execution_table

Concept Snapshot

LangServe API Deployment:
1. Import FastAPI, langserve.add_routes, langchain.chat_models.ChatOpenAI, uvicorn.
2. Create model instance.
3. Create FastAPI app.
4. add_routes(app, model, path="/chat").
5. Start server with uvicorn.run(app).
6. Server handles requests by passing them to model.
7. Responses sent back to clients.

Full Transcript

This visual execution shows how to deploy an API using LangServe with LangChain models. First, import FastAPI, add_routes from langserve, ChatOpenAI, and uvicorn. Create a ChatOpenAI model instance. Create a FastAPI app. Add routes with add_routes(app, model, path="/chat"). Start the server with uvicorn.run(app). When a request comes to /chat/invoke, the server passes it to the model, which processes it and returns a response. The server sends this back to the client. This repeats for each request until stopped.

Practice

1. What is the main purpose of LangServe in LangChain?

easy

A. To quickly turn language models into web APIs

B. To train new language models from scratch

C. To visualize language model outputs in charts

D. To store large datasets for language models

Solution

1. Step 1: Understand LangServe's role

   LangServe is designed to make language models accessible as web APIs easily.

2. Step 2: Compare options with LangServe's function

   Only To quickly turn language models into web APIs matches this purpose; others describe unrelated tasks.

3. Final Answer:

   To quickly turn language models into web APIs -> Option A

4. Quick Check:

   LangServe = API deployment [OK]

Hint: LangServe = language model + web API [OK]

Common Mistakes:

• Confusing LangServe with model training tools
• Thinking LangServe is for data storage
• Assuming LangServe creates visualizations

2. Which of the following is the correct minimal structure for a LangServe class?

easy

A. def MyAPI(input):
return input.upper()

B. class MyAPI:
def __call__(self, input):
return input.upper()

C. class MyAPI:
def call(self, input):
return input.upper()

D. class MyAPI:
def __init__(self, input):
return input.upper()

Solution

1. Step 1: Identify required method for LangServe

   LangServe requires a class with a __call__ method to handle requests.

2. Step 2: Check each option's method name and structure

   Only class MyAPI: def __call__(self, input): return input.upper() uses __call__ correctly; others use wrong method names or invalid return in __init__.

3. Final Answer:

   class with __call__ method -> Option B

4. Quick Check:

   __call__ method = correct structure [OK]

Hint: LangServe needs __call__, not call or __init__ [OK]

Common Mistakes:

• Using call instead of __call__
• Returning values from __init__ method
• Defining a function instead of a class

3. Given this LangServe class:

class EchoAPI:
    def __call__(self, input):
        return f"Echo: {input}"

What will be the output when calling EchoAPI()('hello')?

medium

A. "hello"

B. TypeError: 'EchoAPI' object is not callable

C. "Echo: hello"

D. "EchoAPI: hello"

Solution

1. Step 1: Understand __call__ method behavior

   The __call__ method formats the input by prefixing 'Echo: ' to it.

2. Step 2: Evaluate the call EchoAPI()('hello')

   Creating EchoAPI instance and calling it with 'hello' returns 'Echo: hello'.

3. Final Answer:

   "Echo: hello" -> Option C

4. Quick Check:

   __call__ returns formatted string [OK]

Hint: Calling instance runs __call__ method [OK]

Common Mistakes:

• Expecting raw input without prefix
• Thinking instance is not callable
• Confusing class name with output

4. What is wrong with this LangServe class?

class BadAPI:
    def call(self, input):
        return input[::-1]

medium

A. The return statement should convert input to uppercase

B. The input slicing syntax is incorrect

C. The class must inherit from a base LangServe class

D. The method should be named __call__, not call

Solution

1. Step 1: Check method name required by LangServe

   LangServe expects a __call__ method to make the class callable.

2. Step 2: Analyze method name in BadAPI

   BadAPI uses call instead of __call__, so it won't work as expected.

3. Final Answer:

   The method should be named __call__, not call -> Option D

4. Quick Check:

   __call__ method required [OK]

Hint: Method must be __call__, not call [OK]

Common Mistakes:

• Using call instead of __call__
• Assuming inheritance is mandatory
• Thinking input slicing is invalid

5. You want to deploy a LangServe API that reverses input text but only if the input is a non-empty string. Which class correctly implements this?

hard

A. class ReverseAPI:
def __call__(self, input):
if input is None or input == "":
return "Empty input"
return input[::-1]

B. class ReverseAPI:
def __call__(self, input):
return input[::-1] if input != None else "Empty input"

C. class ReverseAPI:
def __call__(self, input):
if input == "":
return "Empty input"
else:
return input[::-1]

D. class ReverseAPI:
def __call__(self, input):
if input != "":
return input[::-1]
return "Empty input"

Solution

1. Step 1: Identify conditions for input validation

   We must check if input is None or empty string to handle empty input properly.

2. Step 2: Evaluate each option's condition

   class ReverseAPI: def __call__(self, input): if input is None or input == "": return "Empty input" return input[::-1] checks both None and empty string correctly before reversing input.

3. Final Answer:

   Checks both None and empty string before reversing -> Option A

4. Quick Check:

   Check None and empty string before processing [OK]

Hint: Check None and empty string explicitly [OK]

Common Mistakes:

• Only checking for empty string, missing None
• Using != None instead of is None
• Not handling empty input cases