Source citation in RAG responses in LangChain - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Why is source citation important in Retrieval-Augmented Generation (RAG)?

In RAG systems, the model generates answers based on retrieved documents. Why is it important to include source citations in the generated responses?

A To improve the model's training speed by referencing documents

B To increase the randomness of the generated text

C To reduce the size of the retrieval database

D To provide users with evidence and allow verification of the generated information

💡 Hint

Think about trust and transparency when using external information.

Source citation helps users verify the information and trust the generated answers by showing where the information came from.

❓ Predict Output

intermediate

What is the output of this LangChain snippet adding source citations?

Consider this Python code using LangChain to generate a RAG response with source citations:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

retriever = ...  # pre-configured retriever
llm = OpenAI(temperature=0)

qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever, return_source_documents=True)

query = "What is the capital of France?"
result = qa.run(query)
print(result)

What will print(result) output?

A A list of retrieved documents without any answer

B A dictionary with keys 'result' (answer string) and 'source_documents' (list of documents)

C A string answer only, e.g., 'The capital of France is Paris.'

D An error because 'return_source_documents' is not a valid argument

💡 Hint

Check the run() method behavior in LangChain's RetrievalQA.

The run() method returns only the answer string even if return_source_documents=True. To get source documents, you must call call() or __call__() which returns a dict.

❓ Model Choice

advanced

Which LangChain chain type best supports source citation in RAG?

You want to build a RAG system that returns both the answer and the source documents for citation. Which LangChain chain type should you use?

A RetrievalQA with <code>return_source_documents=True</code>

B LLMChain with a prompt template only

C ConversationChain without retrieval

D Simple SequentialChain without retriever

💡 Hint

Consider which chain integrates retrieval and can return sources.

RetrievalQA supports retrieval and can return source documents when return_source_documents=True is set, enabling source citation.

❓ Metrics

advanced

Which metric best evaluates the quality of source citations in RAG responses?

When assessing how well a RAG system cites sources, which metric is most appropriate?

A Recall of retrieved documents that contain correct information

B BLEU score comparing generated text to reference answers

C Perplexity of the language model on generated text

D Training loss of the underlying LLM

💡 Hint

Think about how well the system finds relevant documents for citation.

Recall measures how many relevant documents are retrieved, which directly impacts the quality of source citations in RAG.

🔧 Debug

expert

Why does this LangChain RAG code fail to show source citations?

Given this code snippet:

from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

retriever = ...
llm = OpenAI(temperature=0)
qa = RetrievalQA.from_chain_type(llm=llm, retriever=retriever)

query = "Explain photosynthesis"
result = qa(query)
print(result)

The output shows only the answer text without any source citations. What is the most likely reason?

A The retriever is not connected to any documents

B The <code>return_source_documents</code> parameter was not set to True when creating the RetrievalQA chain

C The OpenAI LLM does not support source citation

D The query string is too short to retrieve documents

💡 Hint

Check the chain initialization parameters related to source documents.

By default, RetrievalQA does not return source documents unless return_source_documents=True is set.