Source citation in RAG responses in LangChain - Step-by-Step Execution

Concept Flow - Source citation in RAG responses

User Query

↓

Retrieve Relevant Docs

↓

Generate Answer Using Docs

↓

Attach Source Citations

↓

Return Answer + Sources

The system takes a user question, finds related documents, creates an answer using those documents, then adds citations showing where info came from.

Execution Sample

LangChain

from langchain.chains import RetrievalQA
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)
response = qa.run("What is LangChain?")
print(response)

This code runs a retrieval-augmented generation query and prints the answer with source citations.

Execution Table

Step	Action	Input	Output	Notes
1	Receive user query	"What is LangChain?"	Query stored	User question ready for retrieval
2	Retrieve documents	Query	List of relevant docs	Documents related to LangChain found
3	Generate answer	Query + docs	Answer text	Answer created using retrieved docs
4	Attach citations	Answer + docs	Answer with citations	Sources referenced in answer
5	Return response	Answer with citations	Displayed to user	Final output includes sources

💡 Process ends after returning answer with source citations to user

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
query	None	"What is LangChain?"	"What is LangChain?"	"What is LangChain?"	"What is LangChain?"
retrieved_docs	[]	[Doc1, Doc2]	[Doc1, Doc2]	[Doc1, Doc2]	[Doc1, Doc2]
answer_text	None	None	"LangChain is a framework..."	"LangChain is a framework... [1][2]"	"LangChain is a framework... [1][2]"

Key Moments - 2 Insights

Why do we need to attach source citations after generating the answer?

Because the answer is created from multiple documents, citations show exactly where each piece of info came from, improving trust and transparency. See execution_table step 4.

What happens if no relevant documents are found during retrieval?

The retrieved_docs list will be empty, so the answer generation may fallback to a default or say no info found. This is shown in variable_tracker after step 2.

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what inputs are used to generate the answer?

A Only the user query

B Only the retrieved documents

C Both the user query and retrieved documents

D Neither query nor documents

💡 Hint

Check the 'Input' column in execution_table row for step 3

According to variable_tracker, what is the value of 'answer_text' after step 4?

A None

B "LangChain is a framework..."

C "LangChain is a framework... [1][2]"

D Empty string

💡 Hint

Look at the 'answer_text' row under 'After Step 4' in variable_tracker

If the retrieved_docs list is empty after step 2, what is likely to happen?

A Answer will be generated without sources

B Answer generation will fail immediately

C Answer will include unrelated sources

D No answer will be returned

💡 Hint

Refer to key_moments about retrieval failure and variable_tracker after step 2

Concept Snapshot

Source citation in RAG responses:
1. Receive user query
2. Retrieve relevant documents
3. Generate answer using docs
4. Attach source citations to answer
5. Return answer with sources
This improves trust by showing info origins.

Full Transcript

This visual execution shows how a retrieval-augmented generation system handles source citation. First, it receives a user query. Then it retrieves documents related to that query. Next, it generates an answer using those documents. After that, it attaches citations referencing the sources used. Finally, it returns the answer with source citations to the user. Variables like the query, retrieved documents, and answer text change step by step. Key moments include why citations are needed and what happens if no documents are found. The quiz tests understanding of inputs, variable values, and fallback behavior.

Practice

1. What is the main purpose of source citation in Retrieval-Augmented Generation (RAG) responses?

easy

A. To show where the AI's answers come from, making them trustworthy

B. To speed up the AI's response time

C. To reduce the size of the AI model

D. To improve the AI's grammar and spelling

Solution

1. Step 1: Understand the role of source citation

   Source citation provides the origin of information used by the AI, helping users trust the answers.

2. Step 2: Compare options with the purpose

   Only To show where the AI's answers come from, making them trustworthy correctly states that citation shows where answers come from to build trust.

3. Final Answer:

   To show where the AI's answers come from, making them trustworthy -> Option A

4. Quick Check:

   Source citation = trust [OK]

Hint: Source citation means showing where info comes from [OK]

Common Mistakes:

• Thinking citation speeds up response
• Confusing citation with model size reduction
• Assuming citation fixes grammar

2. Which Langchain parameter must be set to True to receive source documents in RetrievalQA?

easy

A. return_source_documents

B. include_metadata

C. show_sources

D. enable_citation

Solution

1. Step 1: Recall Langchain RetrievalQA parameters

   The parameter to get source documents is return_source_documents.

2. Step 2: Match options with correct parameter

   Only return_source_documents matches the exact parameter name used in Langchain.

3. Final Answer:

   return_source_documents -> Option A

4. Quick Check:

   Parameter for sources = return_source_documents [OK]

Hint: Look for 'return_source_documents=True' in docs [OK]

Common Mistakes:

• Using incorrect parameter names like include_metadata
• Assuming show_sources or enable_citation exist
• Confusing with unrelated parameters

3. Given this Langchain code snippet, what will print(sources[0].metadata['source']) output if return_source_documents=True?

qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)
result = qa({"query": "What is AI?"})
sources = result['source_documents']
print(sources[0].metadata['source'])

medium

A. The entire content of the first source document

B. The AI's answer to the question

C. An error because 'source' key does not exist

D. The file or URL where the first source document came from

Solution

1. Step 1: Understand what source_documents contain

   They hold documents retrieved with metadata including 'source' indicating origin like file or URL.

2. Step 2: Analyze the print statement

   Accessing metadata['source'] prints the origin of the first document, not the answer or full content.

3. Final Answer:

   The file or URL where the first source document came from -> Option D

4. Quick Check:

   metadata['source'] = document origin [OK]

Hint: metadata['source'] holds origin info, not answer text [OK]

Common Mistakes:

• Thinking it prints AI answer text
• Expecting full document content
• Assuming 'source' key is missing causing error

4. You set return_source_documents=True but get a KeyError when accessing result['source_documents']. What is the likely cause?

medium

A. The retriever has no documents indexed

B. You forgot to set return_source_documents=True

C. The RetrievalQA run method returns a string, not a dict

D. The LLM model is incompatible with source documents

Solution

1. Step 1: Check what qa.run returns

   The run method returns only the answer string, not a dict with sources.

2. Step 2: Identify correct method to get sources

   To get sources, use qa({'query': '...'}) which returns a dict including source_documents.

3. Final Answer:

   The RetrievalQA run method returns a string, not a dict -> Option C

4. Quick Check:

   run() returns string, not dict [OK]

Hint: Use qa({'query': ...}) to get dict with sources, not run() [OK]

Common Mistakes:

• Assuming run() returns dict with sources
• Forgetting to set return_source_documents=True
• Blaming retriever or LLM for KeyError

5. You want to display clear source citations alongside AI answers in a chat app using Langchain RetrievalQA. Which approach correctly combines answer and source snippets for user display?

hard

A. Manually search documents outside Langchain and append sources to answer

B. Run result = qa({'query': question}), then print answer and loop over result['source_documents'] to show metadata['source'] and content snippet

C. Set return_source_documents=False and print the answer with no sources

D. Call qa.run(question) and print the answer only, ignoring sources

Solution

1. Step 1: Use the correct method to get answer and sources

   Calling qa({'query': question}) returns a dict with answer and source documents.

2. Step 2: Format output for user

   Print the answer, then loop over source_documents to show each source's metadata and a snippet of content for clear citation.

3. Final Answer:

   Run result = qa({'query': question}), then print answer and loop over sources for metadata and snippet -> Option B

4. Quick Check:

   Use qa({'query': ...}) + show sources [OK]

Hint: Use qa({'query': ...}) and print sources with snippets [OK]

Common Mistakes:

• Using qa.run() which returns only answer string
• Ignoring source documents in output
• Trying to get sources outside Langchain