Source citation in RAG responses in LangChain - Mini Project: Build & Apply

Source citation in RAG responses

📖 Scenario: You are building a simple Retrieval-Augmented Generation (RAG) system using LangChain. This system answers questions by retrieving relevant documents and then generating answers that include citations to the sources.

🎯 Goal: Create a LangChain script that retrieves documents for a query and generates an answer that includes source citations from the retrieved documents.

📋 What You'll Learn

Create a list called documents with three strings representing document texts.

Create a variable called query with the exact string 'What is LangChain?'.

Use LangChain's FAISS vector store to index the documents.

Create a retriever from the FAISS index with k=2 to get top 2 documents.

Use LangChain's RetrievalQA chain with a dummy LLM that returns a fixed answer including citations.

Generate an answer for the query that includes citations from the retrieved documents.

💡 Why This Matters

🌍 Real World

RAG systems are used in chatbots, virtual assistants, and search engines to provide accurate answers with references to trusted sources.

💼 Career

Understanding how to build RAG pipelines with source citation is valuable for AI developers, data scientists, and software engineers working on intelligent applications.

1

DATA SETUP: Create documents and query

Create a list called documents with these exact strings: 'LangChain is a framework for building applications with LLMs.', 'It helps connect LLMs with external data sources.', and 'Source citation is important in RAG systems.'. Also create a variable called query with the exact string 'What is LangChain?'.

LangChain

# Create the documents list and query variable
# Your code here

Hint

Use a Python list for documents and assign the exact strings. Then assign the exact string to query.

2

CONFIGURATION: Create FAISS index and retriever

Import FAISS and OpenAIEmbeddings from LangChain. Create an embeddings variable using OpenAIEmbeddings(). Then create a FAISS index called index by calling FAISS.from_texts(documents, embeddings). Finally, create a retriever called retriever from index.as_retriever() with search_kwargs={'k': 2}.

LangChain

documents = [
    'LangChain is a framework for building applications with LLMs.',
    'It helps connect LLMs with external data sources.',
    'Source citation is important in RAG systems.'
]
query = 'What is LangChain?'

# Import FAISS and OpenAIEmbeddings
# Create embeddings variable
# Create FAISS index from documents
# Create retriever with k=2
# Your code here

Hint

Import the classes, create embeddings, then create the FAISS index and retriever with k=2.

3

CORE LOGIC: Create RetrievalQA chain with dummy LLM

Import RetrievalQA from LangChain. Create a dummy LLM class called DummyLLM with a __call__ method that takes prompt and returns the string 'LangChain is a framework for LLM apps. [Sources: Doc1, Doc2]'. Then create a llm instance of DummyLLM. Finally, create a qa variable by calling RetrievalQA.from_chain_type(llm=llm, retriever=retriever).

LangChain

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings

# Define DummyLLM class with __call__ method
# Create llm instance
# Import RetrievalQA
# Create qa chain with llm and retriever
# Your code here

Hint

Create a dummy LLM class that returns a fixed answer with citations. Then create the RetrievalQA chain using this LLM and the retriever.

4

COMPLETION: Generate answer with citations

Call qa.run(query) and assign the result to a variable called answer. This will generate the answer including source citations.

LangChain

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA

class DummyLLM:
    def __call__(self, prompt):
        return 'LangChain is a framework for LLM apps. [Sources: Doc1, Doc2]'


# Generate the answer for the query using qa.run
# Your code here

Hint

Use qa.run(query) to get the answer with citations and assign it to answer.

Practice

1. What is the main purpose of source citation in Retrieval-Augmented Generation (RAG) responses?

easy

A. To show where the AI's answers come from, making them trustworthy

B. To speed up the AI's response time

C. To reduce the size of the AI model

D. To improve the AI's grammar and spelling

Solution

1. Step 1: Understand the role of source citation

   Source citation provides the origin of information used by the AI, helping users trust the answers.

2. Step 2: Compare options with the purpose

   Only To show where the AI's answers come from, making them trustworthy correctly states that citation shows where answers come from to build trust.

3. Final Answer:

   To show where the AI's answers come from, making them trustworthy -> Option A

4. Quick Check:

   Source citation = trust [OK]

Hint: Source citation means showing where info comes from [OK]

Common Mistakes:

• Thinking citation speeds up response
• Confusing citation with model size reduction
• Assuming citation fixes grammar

2. Which Langchain parameter must be set to True to receive source documents in RetrievalQA?

easy

A. return_source_documents

B. include_metadata

C. show_sources

D. enable_citation

Solution

1. Step 1: Recall Langchain RetrievalQA parameters

   The parameter to get source documents is return_source_documents.

2. Step 2: Match options with correct parameter

   Only return_source_documents matches the exact parameter name used in Langchain.

3. Final Answer:

   return_source_documents -> Option A

4. Quick Check:

   Parameter for sources = return_source_documents [OK]

Hint: Look for 'return_source_documents=True' in docs [OK]

Common Mistakes:

• Using incorrect parameter names like include_metadata
• Assuming show_sources or enable_citation exist
• Confusing with unrelated parameters

3. Given this Langchain code snippet, what will print(sources[0].metadata['source']) output if return_source_documents=True?

qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)
result = qa({"query": "What is AI?"})
sources = result['source_documents']
print(sources[0].metadata['source'])

medium

A. The entire content of the first source document

B. The AI's answer to the question

C. An error because 'source' key does not exist

D. The file or URL where the first source document came from

Solution

1. Step 1: Understand what source_documents contain

   They hold documents retrieved with metadata including 'source' indicating origin like file or URL.

2. Step 2: Analyze the print statement

   Accessing metadata['source'] prints the origin of the first document, not the answer or full content.

3. Final Answer:

   The file or URL where the first source document came from -> Option D

4. Quick Check:

   metadata['source'] = document origin [OK]

Hint: metadata['source'] holds origin info, not answer text [OK]

Common Mistakes:

• Thinking it prints AI answer text
• Expecting full document content
• Assuming 'source' key is missing causing error

4. You set return_source_documents=True but get a KeyError when accessing result['source_documents']. What is the likely cause?

medium

A. The retriever has no documents indexed

B. You forgot to set return_source_documents=True

C. The RetrievalQA run method returns a string, not a dict

D. The LLM model is incompatible with source documents

Solution

1. Step 1: Check what qa.run returns

   The run method returns only the answer string, not a dict with sources.

2. Step 2: Identify correct method to get sources

   To get sources, use qa({'query': '...'}) which returns a dict including source_documents.

3. Final Answer:

   The RetrievalQA run method returns a string, not a dict -> Option C

4. Quick Check:

   run() returns string, not dict [OK]

Hint: Use qa({'query': ...}) to get dict with sources, not run() [OK]

Common Mistakes:

• Assuming run() returns dict with sources
• Forgetting to set return_source_documents=True
• Blaming retriever or LLM for KeyError

5. You want to display clear source citations alongside AI answers in a chat app using Langchain RetrievalQA. Which approach correctly combines answer and source snippets for user display?

hard

A. Manually search documents outside Langchain and append sources to answer

B. Run result = qa({'query': question}), then print answer and loop over result['source_documents'] to show metadata['source'] and content snippet

C. Set return_source_documents=False and print the answer with no sources

D. Call qa.run(question) and print the answer only, ignoring sources

Solution

1. Step 1: Use the correct method to get answer and sources

   Calling qa({'query': question}) returns a dict with answer and source documents.

2. Step 2: Format output for user

   Print the answer, then loop over source_documents to show each source's metadata and a snippet of content for clear citation.

3. Final Answer:

   Run result = qa({'query': question}), then print answer and loop over sources for metadata and snippet -> Option B

4. Quick Check:

   Use qa({'query': ...}) + show sources [OK]

Hint: Use qa({'query': ...}) and print sources with snippets [OK]

Common Mistakes:

• Using qa.run() which returns only answer string
• Ignoring source documents in output
• Trying to get sources outside Langchain