Source citation in RAG responses in LangChain - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is source citation important in Retrieval-Augmented Generation (RAG)?

In RAG systems, the model generates answers based on retrieved documents. Why is it important to include source citations in the generated responses?

A To improve the model's training speed by referencing documents

B To increase the randomness of the generated text

C To reduce the size of the retrieval database

D To provide users with evidence and allow verification of the generated information

💡 Hint

Think about trust and transparency when using external information.

Source citation helps users verify the information and trust the generated answers by showing where the information came from.

❓ Predict Output

intermediate

What is the output of this LangChain snippet adding source citations?

Consider this Python code using LangChain to generate a RAG response with source citations:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

retriever = ...  # pre-configured retriever
llm = OpenAI(temperature=0)

qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)

query = "What is the capital of France?"
result = qa.run(query)
print(result)

What will print(result) output?

A A list of retrieved documents without any answer

B A dictionary with keys 'result' (answer string) and 'source_documents' (list of documents)

C A string answer only, e.g., 'The capital of France is Paris.'

D An error because 'return_source_documents' is not a valid argument

💡 Hint

Check the run() method behavior in LangChain's RetrievalQA.

The run() method returns only the answer string even if return_source_documents=True. To get source documents, you must call call() or __call__() which returns a dict.

❓ Model Choice

advanced

Which LangChain chain type best supports source citation in RAG?

You want to build a RAG system that returns both the answer and the source documents for citation. Which LangChain chain type should you use?

A RetrievalQA with <code>return_source_documents=True</code>

B LLMChain with a prompt template only

C ConversationChain without retrieval

D Simple SequentialChain without retriever

💡 Hint

Consider which chain integrates retrieval and can return sources.

RetrievalQA supports retrieval and can return source documents when return_source_documents=True is set, enabling source citation.

❓ Metrics

advanced

Which metric best evaluates the quality of source citations in RAG responses?

When assessing how well a RAG system cites sources, which metric is most appropriate?

A Recall of retrieved documents that contain correct information

B BLEU score comparing generated text to reference answers

C Perplexity of the language model on generated text

D Training loss of the underlying LLM

💡 Hint

Think about how well the system finds relevant documents for citation.

Recall measures how many relevant documents are retrieved, which directly impacts the quality of source citations in RAG.

🔧 Debug

expert

Why does this LangChain RAG code fail to show source citations?

Given this code snippet:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

retriever = ...
llm = OpenAI(temperature=0)
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)

query = "Explain photosynthesis"
result = qa(query)
print(result)

The output shows only the answer text without any source citations. What is the most likely reason?

A The retriever is not connected to any documents

B The <code>return_source_documents</code> parameter was not set to True when creating the RetrievalQA chain

C The OpenAI LLM does not support source citation

D The query string is too short to retrieve documents

💡 Hint

Check the chain initialization parameters related to source documents.

By default, RetrievalQA does not return source documents unless return_source_documents=True is set.

Practice

1. What is the main purpose of source citation in Retrieval-Augmented Generation (RAG) responses?

easy

A. To show where the AI's answers come from, making them trustworthy

B. To speed up the AI's response time

C. To reduce the size of the AI model

D. To improve the AI's grammar and spelling

Solution

1. Step 1: Understand the role of source citation

   Source citation provides the origin of information used by the AI, helping users trust the answers.

2. Step 2: Compare options with the purpose

   Only To show where the AI's answers come from, making them trustworthy correctly states that citation shows where answers come from to build trust.

3. Final Answer:

   To show where the AI's answers come from, making them trustworthy -> Option A

4. Quick Check:

   Source citation = trust [OK]

Hint: Source citation means showing where info comes from [OK]

Common Mistakes:

• Thinking citation speeds up response
• Confusing citation with model size reduction
• Assuming citation fixes grammar

2. Which Langchain parameter must be set to True to receive source documents in RetrievalQA?

easy

A. return_source_documents

B. include_metadata

C. show_sources

D. enable_citation

Solution

1. Step 1: Recall Langchain RetrievalQA parameters

   The parameter to get source documents is return_source_documents.

2. Step 2: Match options with correct parameter

   Only return_source_documents matches the exact parameter name used in Langchain.

3. Final Answer:

   return_source_documents -> Option A

4. Quick Check:

   Parameter for sources = return_source_documents [OK]

Hint: Look for 'return_source_documents=True' in docs [OK]

Common Mistakes:

• Using incorrect parameter names like include_metadata
• Assuming show_sources or enable_citation exist
• Confusing with unrelated parameters

3. Given this Langchain code snippet, what will print(sources[0].metadata['source']) output if return_source_documents=True?

qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)
result = qa({"query": "What is AI?"})
sources = result['source_documents']
print(sources[0].metadata['source'])

medium

A. The entire content of the first source document

B. The AI's answer to the question

C. An error because 'source' key does not exist

D. The file or URL where the first source document came from

Solution

1. Step 1: Understand what source_documents contain

   They hold documents retrieved with metadata including 'source' indicating origin like file or URL.

2. Step 2: Analyze the print statement

   Accessing metadata['source'] prints the origin of the first document, not the answer or full content.

3. Final Answer:

   The file or URL where the first source document came from -> Option D

4. Quick Check:

   metadata['source'] = document origin [OK]

Hint: metadata['source'] holds origin info, not answer text [OK]

Common Mistakes:

• Thinking it prints AI answer text
• Expecting full document content
• Assuming 'source' key is missing causing error

4. You set return_source_documents=True but get a KeyError when accessing result['source_documents']. What is the likely cause?

medium

A. The retriever has no documents indexed

B. You forgot to set return_source_documents=True

C. The RetrievalQA run method returns a string, not a dict

D. The LLM model is incompatible with source documents

Solution

1. Step 1: Check what qa.run returns

   The run method returns only the answer string, not a dict with sources.

2. Step 2: Identify correct method to get sources

   To get sources, use qa({'query': '...'}) which returns a dict including source_documents.

3. Final Answer:

   The RetrievalQA run method returns a string, not a dict -> Option C

4. Quick Check:

   run() returns string, not dict [OK]

Hint: Use qa({'query': ...}) to get dict with sources, not run() [OK]

Common Mistakes:

• Assuming run() returns dict with sources
• Forgetting to set return_source_documents=True
• Blaming retriever or LLM for KeyError

5. You want to display clear source citations alongside AI answers in a chat app using Langchain RetrievalQA. Which approach correctly combines answer and source snippets for user display?

hard

A. Manually search documents outside Langchain and append sources to answer

B. Run result = qa({'query': question}), then print answer and loop over result['source_documents'] to show metadata['source'] and content snippet

C. Set return_source_documents=False and print the answer with no sources

D. Call qa.run(question) and print the answer only, ignoring sources

Solution

1. Step 1: Use the correct method to get answer and sources

   Calling qa({'query': question}) returns a dict with answer and source documents.

2. Step 2: Format output for user

   Print the answer, then loop over source_documents to show each source's metadata and a snippet of content for clear citation.

3. Final Answer:

   Run result = qa({'query': question}), then print answer and loop over sources for metadata and snippet -> Option B

4. Quick Check:

   Use qa({'query': ...}) + show sources [OK]

Hint: Use qa({'query': ...}) and print sources with snippets [OK]

Common Mistakes:

• Using qa.run() which returns only answer string
• Ignoring source documents in output
• Trying to get sources outside Langchain