Multi-query retrieval for better recall in LangChain - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ component_behavior

intermediate

What is the output of this multi-query retrieval code snippet?

Consider this LangChain code that performs multi-query retrieval to improve recall. What will be the printed output?

LangChain

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.load_local("my_faiss_index", embeddings)

retriever = vectorstore.as_retriever(search_kwargs={"k": 2})

queries = ["What is AI?", "Explain machine learning."]

results = []
for query in queries:
    qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=retriever)
    answer = qa.run(query)
    results.append(answer)

print(results)

A A list of two answer strings, each answering one query

B A single string combining answers to both queries

C A list of two empty strings

D Raises a TypeError because 'retriever' is not iterable

💡 Hint

Think about how the loop runs the retrieval QA chain separately for each query and collects answers.

The code runs a retrieval QA chain for each query separately and collects the answers in a list. So the output is a list of two answer strings, one per query.

📝 Syntax

intermediate

Which option correctly initializes a multi-query retriever with LangChain?

Choose the correct code snippet that sets up a retriever to handle multiple queries with LangChain's FAISS vector store.

A retriever = vectorstore.as_retriever(k=3)

B retriever = vectorstore.as_retriever(search_k=3)

C retriever = vectorstore.as_retriever({"k": 3})

D retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

💡 Hint

Check the correct parameter name for passing search options in LangChain retrievers.

The correct way is to pass search parameters inside a dictionary to the 'search_kwargs' argument. So option D is correct.

🔧 Debug

advanced

Why does this multi-query retrieval code raise an error?

Given this code snippet, why does it raise an error?

LangChain

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.load_local("index_dir", embeddings)

queries = ["Define AI", "What is NLP?"]

retriever = vectorstore.as_retriever(search_kwargs={"k": 2})

answers = [retriever.run(query) for query in queries]
print(answers)

A TypeError because queries is not iterable

B SyntaxError due to missing colon

C 'Retriever' object has no attribute 'run' error

D No error, prints list of answers

💡 Hint

Check if the retriever object supports a 'run' method.

The retriever object does not have a 'run' method. It is used to fetch documents, not to answer queries directly. You need to use a chain like RetrievalQA to get answers.

❓ state_output

advanced

What is the value of 'combined_answers' after running this multi-query retrieval code?

Analyze the code and determine the value of the variable combined_answers after execution.

LangChain

from langchain.vectorstores import FAISS
from langchain.embeddings import OpenAIEmbeddings
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI

embeddings = OpenAIEmbeddings()
vectorstore = FAISS.load_local("faiss_index", embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 1})

queries = ["Explain AI", "What is deep learning?"]

answers = []
for q in queries:
    qa_chain = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=retriever)
    answers.append(qa_chain.run(q))

combined_answers = " | ".join(answers)

A A list of empty strings

B A single string combining two answers separated by ' | '

C An empty string

D A list of two answer strings

💡 Hint

Look at how the answers list is joined into a single string.

The answers list contains two strings. Joining them with ' | ' creates one combined string with answers separated by ' | '.

🧠 Conceptual

expert

Why use multi-query retrieval in LangChain for better recall?

Which reason best explains why multi-query retrieval improves recall in LangChain applications?

A It allows querying multiple related questions separately to gather more relevant documents

B It speeds up retrieval by running queries in parallel

C It reduces the number of documents retrieved to save memory

D It merges all queries into one to simplify processing

💡 Hint

Think about how splitting a complex question into parts affects document retrieval.

Multi-query retrieval splits a complex information need into multiple queries, each retrieving relevant documents. This increases the chance of finding all useful information, improving recall.

Practice

1. What is the main purpose of multi-query retrieval in langchain?

easy

A. To store data in multiple vector stores

B. To reduce the number of queries sent to the database

C. To speed up the response time by using a single query

D. To ask several related questions to get better search results

Solution

1. Step 1: Understand the concept of multi-query retrieval

   Multi-query retrieval involves asking multiple related questions instead of just one to improve the quality of search results.

2. Step 2: Identify the main goal of this approach

   The goal is to get better and more complete search results by combining answers from several queries.

3. Final Answer:

   To ask several related questions to get better search results -> Option D

4. Quick Check:

   Multi-query retrieval = multiple related questions for better results [OK]

Hint: Think: multiple questions improve search quality [OK]

Common Mistakes:

• Confusing multi-query with reducing query count
• Thinking it stores data in multiple places
• Assuming it speeds up by using fewer queries

2. Which of the following is the correct way to create a multi-query retriever in langchain?

easy

A. MultiQueryRetriever(vectorstore, llm=llm)

B. MultiQueryRetriever(queries=["query1", "query2"])

C. MultiQueryRetriever.from_llm(vectorstore.as_retriever(), llm)

D. MultiQueryRetriever.from_llm(vectorstore, llm)

Solution

1. Step 1: Check the required parameters for MultiQueryRetriever

   The MultiQueryRetriever is created using from_llm which requires a BaseRetriever (from vectorstore.as_retriever()) and an llm.

2. Step 2: Identify the correct syntax

   MultiQueryRetriever.from_llm(vectorstore.as_retriever(), llm) correctly uses the classmethod with proper arguments.

3. Final Answer:

   MultiQueryRetriever.from_llm(vectorstore.as_retriever(), llm) -> Option C

4. Quick Check:

   Correct syntax includes from_llm, .as_retriever(), and llm [OK]

Hint: Look for from_llm(vectorstore.as_retriever(), llm) syntax [OK]

Common Mistakes:

• Passing queries as a list to constructor
• Using direct constructor without llm_chain
• Forgetting .as_retriever() on vectorstore

3. Given the following code snippet, what will be the output type of results?

retriever = MultiQueryRetriever.from_llm(vectorstore.as_retriever(), llm)
results = retriever.get_relevant_documents("AI")

medium

A. A single document matching the first query only

B. A list of documents combining results from both queries

C. A dictionary with queries as keys and documents as values

D. An error because get_relevant_documents requires no arguments

Solution

1. Step 1: Understand MultiQueryRetriever behavior

   It [uses the LLM to] runs multiple queries and combines their results into a single list of documents.

2. Step 2: Analyze the method call

   Calling get_relevant_documents("AI") triggers retrieval for all [generated] queries, returning combined documents.

3. Final Answer:

   A list of documents combining results from both queries -> Option B

4. Quick Check:

   Multi-query retrieval returns combined document list [OK]

Hint: MultiQueryRetriever returns combined list of documents [OK]

Common Mistakes:

• Expecting a dictionary instead of a list
• Thinking it returns only first query results
• Assuming get_relevant_documents takes no arguments

4. Identify the error in this code snippet using multi-query retrieval:

retriever = MultiQueryRetriever.from_llm(vectorstore, llm)
results = retriever.get_relevant_documents("AI")

medium

A. vectorstore.as_retriever() is missing

B. get_relevant_documents requires a query argument

C. vectorstore is not defined

D. MultiQueryRetriever does not exist in langchain

Solution

1. Step 1: Check the constructor parameters

   from_llm expects a BaseRetriever as the first argument, but vectorstore is a VectorStore; use vectorstore.as_retriever().

2. Step 2: Verify other parts of the code

   get_relevant_documents has argument; llm assumed defined; class exists.

3. Final Answer:

   vectorstore.as_retriever() is missing -> Option A

4. Quick Check:

   from_llm requires BaseRetriever [OK]

Hint: from_llm needs vectorstore.as_retriever() [OK]

Common Mistakes:

• Passing vectorstore directly instead of retriever
• Thinking get_relevant_documents needs no arg when it does
• Assuming vectorstore undefined without context

5. You want to improve recall for a complex topic by using multi-query retrieval with three related queries. Which approach best ensures you get the most relevant documents without duplicates?

hard

A. Use MultiQueryRetriever.from_llm(retriever, llm) to generate multiple queries and deduplicate results

B. Run each query separately and pick the first result from each

C. Combine all queries into one long string and run a single query

D. Use only the query with the highest expected relevance

Solution

1. Step 1: Understand multi-query retrieval goal

   It aims to combine results from multiple related queries to improve recall.

2. Step 2: Identify best practice to avoid duplicates

   Using MultiQueryRetriever.from_llm(retriever, llm) generates multiple queries via LLM and deduplicates results for completeness without repeats.

3. Final Answer:

   Use MultiQueryRetriever.from_llm(retriever, llm) to generate multiple queries and deduplicate results -> Option A

4. Quick Check:

   MultiQueryRetriever generates + deduplicates = best recall [OK]

Hint: MultiQueryRetriever generates queries then deduplicates for best recall [OK]

Common Mistakes:

• Running queries separately without combining results
• Merging queries into one string loses query specificity
• Using only one query misses relevant documents