Retrieval strategies (similarity, MMR, hybrid) in Agentic AI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Similarity-Based Retrieval

Which statement best describes how similarity-based retrieval works in information retrieval systems?

A It retrieves documents by measuring how closely their content matches the query using a similarity score.

B It randomly selects documents without considering the query content.

C It ranks documents solely based on their length compared to the query length.

D It retrieves documents based on the time they were added to the database.

💡 Hint

Think about how a search engine finds documents that are most relevant to what you type.

Similarity-based retrieval measures how close the content of documents is to the query, often using scores like cosine similarity, to rank and retrieve the most relevant documents.

❓ Model Choice

intermediate

Choosing a Retrieval Strategy for Diverse Results

You want to retrieve documents that are not only relevant but also diverse to avoid redundancy. Which retrieval strategy should you choose?

A Maximal Marginal Relevance (MMR)

B Retrieval based on document age

C Random sampling of documents

D Pure similarity-based retrieval

💡 Hint

Consider a method that balances relevance and diversity.

MMR balances relevance to the query and diversity among selected documents to reduce redundancy in results.

❓ Predict Output

advanced

Output of MMR Score Calculation

Given the following Python code snippet that calculates MMR scores for candidate documents, what is the output of the print statement?

Agentic AI

query_vec = [1, 0]
candidate_vecs = [[0.9, 0.1], [0.1, 0.9], [0.8, 0.2]]
selected = []

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

lambda_param = 0.5

mmr_scores = []
for i, doc_vec in enumerate(candidate_vecs):
    sim_to_query = cosine_similarity(query_vec, doc_vec)
    sim_to_selected = max([cosine_similarity(doc_vec, candidate_vecs[j]) for j in selected] or [0])
    score = lambda_param * sim_to_query - (1 - lambda_param) * sim_to_selected
    mmr_scores.append(score)

print([round(s, 2) for s in mmr_scores])

A [0.5, 0.5, 0.5]

B [0.95, 0.05, 0.9]

C [0.5, -0.5, 0.5]

D [0.9, -0.05, 0.8]

💡 Hint

Calculate cosine similarity carefully and apply the MMR formula.

The first document has high similarity to the query and no penalty since no documents are selected yet, so score is 0.5*0.9 - 0.5*0 = 0.45 rounded to 0.45 (not 0.9). The others have lower scores due to similarity and penalty terms.

❓ Hyperparameter

advanced

Effect of Lambda in MMR

In Maximal Marginal Relevance (MMR), what happens when the lambda parameter is set close to 1?

A The retrieval balances relevance and diversity equally.

B The retrieval focuses mostly on relevance, ignoring diversity.

C The retrieval randomly selects documents without any scoring.

D The retrieval focuses mostly on document diversity, ignoring relevance.

💡 Hint

Lambda controls the trade-off between relevance and diversity.

Lambda near 1 means the formula weights relevance heavily and diversity very little, so results are mostly relevant but may be redundant.

🔧 Debug

expert

Debugging Hybrid Retrieval Code

Consider this hybrid retrieval code combining similarity and MMR. What error will it raise when run?

Agentic AI

def hybrid_retrieval(query_vec, candidate_vecs, lambda_param=0.7):
    import numpy as np
    def cosine_similarity(a, b):
        return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

    selected = []
    scores = []
    for i, doc_vec in enumerate(candidate_vecs):
        sim_query = cosine_similarity(query_vec, doc_vec)
        sim_selected = max([cosine_similarity(doc_vec, candidate_vecs[j]) for j in selected] or [0])
        score = lambda_param * sim_query - (1 - lambda_param) * sim_selected
        scores.append(score)
    return scores

query = [1, 0]
candidates = [[0.9, 0.1], [0.1, 0.9]]
print(hybrid_retrieval(query, candidates))

A ZeroDivisionError due to zero vector norm

B TypeError due to unsupported operand types

C ValueError because max() arg is an empty sequence

D No error, outputs a list of scores

💡 Hint

Look at the max() function call when selected is empty.

When selected is empty, the list comprehension inside max() is empty, causing max() to raise a ValueError for empty sequence.

Practice

1. Which retrieval strategy focuses on ranking results purely based on how close they are to the query?

easy

A. Random retrieval

B. Maximal Marginal Relevance (MMR)

C. Similarity-based retrieval

D. Hybrid retrieval

Solution

1. Step 1: Understand similarity-based retrieval

   Similarity-based retrieval ranks results by how close or similar they are to the query, focusing only on relevance.

2. Step 2: Compare with other strategies

   MMR balances relevance and diversity, hybrid combines methods, and random is unrelated.

3. Final Answer:

   Similarity-based retrieval -> Option C

4. Quick Check:

   Similarity = closeness only [OK]

Hint: Similarity means closest match only [OK]

Common Mistakes:

• Confusing MMR with similarity
• Thinking hybrid is only similarity
• Choosing random as a valid strategy

2. Which of the following is the correct way to describe Maximal Marginal Relevance (MMR)?

easy

A. Combines all retrieval methods without weighting

B. Ranks results by random selection

C. Only uses keyword matching

D. Balances relevance and diversity in retrieval

Solution

1. Step 1: Define MMR

   MMR is designed to balance relevance to the query and diversity among the results to avoid redundancy.

2. Step 2: Eliminate incorrect options

   Random selection is unrelated, keyword matching is too narrow, and combining without weighting is not MMR.

3. Final Answer:

   Balances relevance and diversity in retrieval -> Option D

4. Quick Check:

   MMR = relevance + diversity [OK]

Hint: MMR mixes relevance with diversity [OK]

Common Mistakes:

• Thinking MMR is random
• Assuming MMR uses only keywords
• Believing MMR combines methods blindly

3. Given the following pseudo-code for a hybrid retrieval method combining similarity and MMR scores:

results = []
for doc in documents:
    sim_score = similarity(query, doc)
    mmr_score = mmr(query, doc, results)
    combined_score = 0.6 * sim_score + 0.4 * mmr_score
    results.append((doc, combined_score))
results.sort(key=lambda x: x[1], reverse=True)
print([doc for doc, score in results[:3]])

What does this code output?

medium

A. Top 3 documents ranked by combined similarity and MMR scores

B. Top 3 documents ranked by similarity score only

C. Top 3 documents ranked by MMR score only

D. Random 3 documents from the list

Solution

1. Step 1: Analyze score calculation

   The code calculates a combined score using 60% similarity and 40% MMR for each document.

2. Step 2: Understand sorting and output

   Documents are sorted by this combined score in descending order, then top 3 are printed.

3. Final Answer:

   Top 3 documents ranked by combined similarity and MMR scores -> Option A

4. Quick Check:

   Hybrid = combined scores [OK]

Hint: Check weighted sum and sorting for final ranking [OK]

Common Mistakes:

• Ignoring MMR score in combined score
• Assuming sorting by similarity only
• Thinking output is random

4. Consider this buggy code snippet for MMR retrieval:

def mmr(query, docs, selected):
    scores = []
    for doc in docs:
        relevance = similarity(query, doc)
        diversity = min([similarity(doc, s) for s in selected])
        score = relevance - 0.5 * diversity
        scores.append((doc, score))
    return max(scores, key=lambda x: x[1])[0]

What is the main error causing a crash when selected is empty?

medium

A. Using min() on an empty list causes an error

B. Incorrect use of max() function

C. Missing return statement

D. Similarity function is undefined

Solution

1. Step 1: Identify cause of crash

   When selected is empty, the list inside min() is empty, causing a ValueError.

2. Step 2: Understand min() behavior

   min() cannot operate on empty lists, so the code crashes at that line.

3. Final Answer:

   Using min() on an empty list causes an error -> Option A

4. Quick Check:

   min(empty list) = error [OK]

Hint: Check min() on empty lists for errors [OK]

Common Mistakes:

• Blaming max() instead of min()
• Ignoring empty list edge case
• Assuming similarity is undefined

5. You want to improve a search system by combining similarity and MMR retrieval. Which approach best balances relevance and diversity in the final results?

hard

A. Use MMR with a diversity weight of zero

B. Combine similarity and MMR scores with adjustable weights

C. Use only similarity scores to rank results

D. Randomly shuffle results after similarity ranking

Solution

1. Step 1: Understand the goal

   Balancing relevance and diversity requires combining both similarity and MMR scores meaningfully.

2. Step 2: Evaluate options

   Using only similarity or zero diversity weight ignores diversity; random shuffling loses relevance order.

3. Step 3: Best approach

   Combining similarity and MMR with adjustable weights allows tuning the balance effectively.

4. Final Answer:

   Combine similarity and MMR scores with adjustable weights -> Option B

5. Quick Check:

   Hybrid weighted combination = best balance [OK]

Hint: Adjust weights to balance relevance and diversity [OK]

Common Mistakes:

• Ignoring diversity by using similarity only
• Setting diversity weight to zero in MMR
• Randomizing results without scoring