Retrieval strategies (similarity, MMR, hybrid) in Agentic AI - Deep Dive

Overview - Retrieval strategies (similarity, MMR, hybrid)

What is it?

Retrieval strategies are methods used to find the most relevant information from a large collection based on a query. Similarity-based retrieval finds items closest to the query by comparing features. MMR, or Maximal Marginal Relevance, balances relevance with diversity to avoid repetitive results. Hybrid strategies combine multiple approaches to improve the quality of retrieved information.

Why it matters

Without effective retrieval strategies, systems would return irrelevant or repetitive information, making it hard to find useful answers quickly. Good retrieval helps search engines, chatbots, and AI assistants provide accurate and varied responses, improving user experience and decision-making.

Where it fits

Learners should first understand basic concepts of vectors and similarity measures like cosine similarity. After mastering retrieval strategies, they can explore advanced topics like neural search, ranking algorithms, and reinforcement learning for retrieval optimization.

Mental Model

Core Idea

Retrieval strategies find the best and most diverse information by measuring closeness and balancing relevance with variety.

Think of it like...

Imagine picking fruits from a basket: similarity retrieval picks fruits that look most like your favorite, MMR picks fruits that are both tasty and different from each other, and hybrid strategies mix these ways to get the best basket.

┌───────────────┐
│   Query Input │
└──────┬────────┘
       │
       ▼
┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Similarity    │──────▶│ MMR           │──────▶│ Hybrid        │
│ Retrieval     │       │ Retrieval     │       │ Retrieval     │
└───────────────┘       └───────────────┘       └───────────────┘
       │                      │                      │
       ▼                      ▼                      ▼
┌────────────────────────────────────────────────────────┐
│                Retrieved Results                        │
└────────────────────────────────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding similarity-based retrieval

Concept: Similarity retrieval finds items closest to a query by comparing features using measures like cosine similarity.

Imagine you have a list of documents and a question. Each document and the question are turned into numbers (vectors). We measure how close these vectors are using cosine similarity, which checks the angle between them. The larger the cosine similarity, the more similar they are. We then pick the top documents with the highest similarity scores.

Result

You get a list of documents ranked by how similar they are to your query.

Understanding similarity retrieval is key because it forms the base for most search and recommendation systems.

2

Foundation

Basics of diversity in retrieval

Concept: Diversity ensures retrieved results are not too similar to each other, avoiding repetition.

If you only pick the most similar items, you might get many that say the same thing. Diversity means choosing items that cover different aspects or ideas. This helps users see a broader range of information instead of repeats.

Result

Results include varied information, making the search more useful and interesting.

Recognizing the need for diversity prevents boring or redundant results, improving user satisfaction.

3

Intermediate

Maximal Marginal Relevance (MMR) explained

🤔Before reading on: do you think MMR picks only the most relevant items or also considers how different they are? Commit to your answer.

Concept: MMR balances relevance to the query with diversity among the selected items to avoid redundancy.

MMR works by picking items one by one. Each new item is chosen to be both relevant to the query and different from items already picked. It uses a formula that subtracts similarity to already chosen items from the relevance score, ensuring variety.

Result

The final list is both relevant and diverse, avoiding repeated information.

Knowing MMR helps you understand how to smartly mix relevance and diversity, which is crucial for better search results.

4

Intermediate

Similarity measures beyond cosine

🤔Before reading on: do you think cosine similarity is the only way to measure closeness? Commit to yes or no.

Concept: There are many ways to measure similarity, like Euclidean distance or dot product, each with pros and cons.

Cosine similarity measures angle between vectors, ignoring length. Euclidean distance measures straight-line distance. Dot product considers both angle and length. Choice depends on data and task. For example, cosine is good for text, Euclidean for spatial data.

Result

You can select the best similarity measure for your specific retrieval problem.

Understanding different similarity measures lets you tailor retrieval to your data type and improve accuracy.

5

Intermediate

Hybrid retrieval strategies overview

🤔Before reading on: do you think combining retrieval methods always improves results? Commit to yes or no.

Concept: Hybrid strategies combine similarity and MMR or other methods to leverage strengths of each.

Hybrid retrieval might first use similarity to find a broad set of candidates, then apply MMR to select a diverse subset. Or it can combine scores from different models. This approach balances relevance, diversity, and sometimes speed.

Result

Retrieval results are more balanced, accurate, and useful than using one method alone.

Knowing hybrid strategies shows how combining ideas can solve complex retrieval challenges better.

6

Advanced

Tuning MMR parameters for best results

🤔Before reading on: do you think increasing diversity weight in MMR always improves results? Commit to yes or no.

Concept: MMR uses a parameter to control the trade-off between relevance and diversity, which must be tuned carefully.

The MMR formula has a lambda parameter between 0 and 1. Lambda near 1 favors relevance, near 0 favors diversity. Setting it too low can reduce relevance, too high can cause repetition. Tuning depends on user needs and data.

Result

Proper tuning leads to retrieval results that best match user expectations for relevance and variety.

Understanding parameter tuning prevents poor retrieval quality and helps customize results for different applications.

7

Expert

Surprising effects of hybrid retrieval in practice

🤔Before reading on: do you think hybrid retrieval always outperforms pure similarity or MMR? Commit to yes or no.

Concept: Hybrid retrieval can sometimes underperform if components conflict or are poorly combined, requiring careful design.

In real systems, combining retrieval methods can cause unexpected issues like score scaling mismatches or increased latency. Sometimes simpler methods work better. Experts carefully analyze data distributions and system constraints before choosing hybrids.

Result

Hybrid retrieval is powerful but not a guaranteed improvement; it requires expert tuning and validation.

Knowing the limits and pitfalls of hybrid methods helps avoid wasted effort and ensures robust retrieval system design.

Under the Hood

Similarity retrieval converts data and queries into vectors in a shared space, then calculates distances or angles to rank items. MMR iteratively selects items by scoring relevance minus similarity to already chosen items, balancing novelty and closeness. Hybrid methods combine these steps or scores, often requiring normalization and weighting to integrate different signals.

Why designed this way?

Similarity retrieval was designed for simplicity and efficiency in high-dimensional spaces. MMR was introduced to solve the problem of repetitive results by explicitly adding diversity. Hybrid methods emerged as practitioners realized no single method fits all needs, so combining strengths improves practical performance.

┌───────────────┐
│ Data Vectors  │
└──────┬────────┘
       │
       ▼
┌───────────────┐       ┌───────────────┐
│ Similarity    │──────▶│ Candidate Set │
│ Calculation   │       └──────┬────────┘
└───────────────┘              │
                               ▼
                      ┌─────────────────┐
                      │ MMR Selection   │
                      └──────┬──────────┘
                             │
                             ▼
                    ┌───────────────────┐
                    │ Final Retrieved   │
                    │ Results           │
                    └───────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does MMR always pick the most relevant items first? Commit to yes or no.

Common Belief: MMR just picks the most relevant items in order.

Reality: MMR balances relevance with diversity, so it may skip some highly relevant but similar items to increase variety.

Why it matters: Assuming MMR only cares about relevance can lead to misunderstanding its purpose and misusing it in applications needing diverse results.

Quick: Is cosine similarity sensitive to vector length? Commit to yes or no.

Common Belief: Cosine similarity measures how close vectors are including their length.

Reality: Cosine similarity measures the angle between vectors, ignoring their length, focusing on direction only.

Why it matters: Confusing this can cause wrong similarity calculations, especially when vector magnitude carries important meaning.

Quick: Does combining retrieval methods always improve results? Commit to yes or no.

Common Belief: Hybrid retrieval always outperforms single methods.

Reality: Hybrid retrieval can sometimes degrade performance if methods conflict or are poorly integrated.

Why it matters: Blindly combining methods without tuning wastes resources and can harm user experience.

Quick: Is diversity always beneficial in retrieval? Commit to yes or no.

Common Belief: More diversity always means better retrieval results.

Reality: Too much diversity can reduce relevance and confuse users by showing unrelated items.

Why it matters: Balancing diversity and relevance is crucial; ignoring this leads to poor search quality.

Expert Zone

1

MMR's effectiveness depends heavily on the choice and scaling of similarity measures between items, which is often overlooked.

2

Hybrid retrieval systems must carefully normalize and weight scores from different methods to avoid biasing results toward one approach.

3

Latency and computational cost can increase significantly with hybrid methods, requiring trade-offs between quality and speed.

When NOT to use

Avoid MMR when the user needs strictly the most relevant items without concern for diversity, such as in precise fact retrieval. Hybrid methods may not be suitable for real-time systems with strict latency constraints; simpler similarity retrieval or approximate nearest neighbor search might be better.

Production Patterns

In production, retrieval often uses a two-stage approach: a fast similarity-based candidate generation followed by a reranking stage using MMR or learned models. Hybrid methods are common in AI assistants to balance relevance and coverage. Parameter tuning and monitoring user feedback are standard practices to maintain quality.

Connections

Recommender Systems

Retrieval strategies like MMR are used to balance relevance and diversity in recommendations.

Understanding retrieval diversity helps improve recommendation variety, preventing repetitive suggestions.

Information Theory

MMR's balance of relevance and diversity relates to maximizing information gain and reducing redundancy.

Knowing information theory concepts clarifies why diversity improves the usefulness of retrieved information.

Portfolio Management (Finance)

MMR's trade-off between relevance and diversity is similar to balancing risk and return in investment portfolios.

Recognizing this analogy helps grasp how retrieval strategies optimize multiple competing goals simultaneously.

Common Pitfalls

#1 Ignoring diversity leads to repetitive results.

Wrong approach:

Retrieve top 10 items by similarity score only, without considering overlap.

Correct approach:

Use MMR or diversity-aware methods to select items balancing relevance and variety.

Root cause:

Misunderstanding that highest similarity alone guarantees best user experience.

#2 Using unnormalized scores in hybrid retrieval causes bias.

Wrong approach:

Combine raw similarity and MMR scores by simple addition without scaling.

Correct approach:

Normalize scores from each method before combining to ensure fair weighting.

Root cause:

Overlooking differences in score ranges and distributions between methods.

#3 Setting MMR lambda parameter incorrectly harms results.

Wrong approach:

Set lambda to 0 (full diversity) or 1 (full relevance) without tuning.

Correct approach:

Tune lambda between 0 and 1 based on validation to balance relevance and diversity.

Root cause:

Assuming extreme parameter values always yield best outcomes.

Key Takeaways

Retrieval strategies find relevant information by measuring closeness between query and data items.

MMR improves retrieval by balancing relevance with diversity to avoid repetitive results.

Hybrid retrieval combines multiple methods to leverage their strengths but requires careful tuning.

Choosing the right similarity measure and tuning parameters is crucial for effective retrieval.

Understanding retrieval trade-offs helps design systems that deliver useful and varied information.

Practice

1. Which retrieval strategy focuses on ranking results purely based on how close they are to the query?

easy

A. Random retrieval

B. Maximal Marginal Relevance (MMR)

C. Similarity-based retrieval

D. Hybrid retrieval

Solution

1. Step 1: Understand similarity-based retrieval

   Similarity-based retrieval ranks results by how close or similar they are to the query, focusing only on relevance.

2. Step 2: Compare with other strategies

   MMR balances relevance and diversity, hybrid combines methods, and random is unrelated.

3. Final Answer:

   Similarity-based retrieval -> Option C

4. Quick Check:

   Similarity = closeness only [OK]

Hint: Similarity means closest match only [OK]

Common Mistakes:

• Confusing MMR with similarity
• Thinking hybrid is only similarity
• Choosing random as a valid strategy

2. Which of the following is the correct way to describe Maximal Marginal Relevance (MMR)?

easy

A. Combines all retrieval methods without weighting

B. Ranks results by random selection

C. Only uses keyword matching

D. Balances relevance and diversity in retrieval

Solution

1. Step 1: Define MMR

   MMR is designed to balance relevance to the query and diversity among the results to avoid redundancy.

2. Step 2: Eliminate incorrect options

   Random selection is unrelated, keyword matching is too narrow, and combining without weighting is not MMR.

3. Final Answer:

   Balances relevance and diversity in retrieval -> Option D

4. Quick Check:

   MMR = relevance + diversity [OK]

Hint: MMR mixes relevance with diversity [OK]

Common Mistakes:

• Thinking MMR is random
• Assuming MMR uses only keywords
• Believing MMR combines methods blindly

3. Given the following pseudo-code for a hybrid retrieval method combining similarity and MMR scores:

results = []
for doc in documents:
    sim_score = similarity(query, doc)
    mmr_score = mmr(query, doc, results)
    combined_score = 0.6 * sim_score + 0.4 * mmr_score
    results.append((doc, combined_score))
results.sort(key=lambda x: x[1], reverse=True)
print([doc for doc, score in results[:3]])

What does this code output?

medium

A. Top 3 documents ranked by combined similarity and MMR scores

B. Top 3 documents ranked by similarity score only

C. Top 3 documents ranked by MMR score only

D. Random 3 documents from the list

Solution

1. Step 1: Analyze score calculation

   The code calculates a combined score using 60% similarity and 40% MMR for each document.

2. Step 2: Understand sorting and output

   Documents are sorted by this combined score in descending order, then top 3 are printed.

3. Final Answer:

   Top 3 documents ranked by combined similarity and MMR scores -> Option A

4. Quick Check:

   Hybrid = combined scores [OK]

Hint: Check weighted sum and sorting for final ranking [OK]

Common Mistakes:

• Ignoring MMR score in combined score
• Assuming sorting by similarity only
• Thinking output is random

4. Consider this buggy code snippet for MMR retrieval:

def mmr(query, docs, selected):
    scores = []
    for doc in docs:
        relevance = similarity(query, doc)
        diversity = min([similarity(doc, s) for s in selected])
        score = relevance - 0.5 * diversity
        scores.append((doc, score))
    return max(scores, key=lambda x: x[1])[0]

What is the main error causing a crash when selected is empty?

medium

A. Using min() on an empty list causes an error

B. Incorrect use of max() function

C. Missing return statement

D. Similarity function is undefined

Solution

1. Step 1: Identify cause of crash

   When selected is empty, the list inside min() is empty, causing a ValueError.

2. Step 2: Understand min() behavior

   min() cannot operate on empty lists, so the code crashes at that line.

3. Final Answer:

   Using min() on an empty list causes an error -> Option A

4. Quick Check:

   min(empty list) = error [OK]

Hint: Check min() on empty lists for errors [OK]

Common Mistakes:

• Blaming max() instead of min()
• Ignoring empty list edge case
• Assuming similarity is undefined

5. You want to improve a search system by combining similarity and MMR retrieval. Which approach best balances relevance and diversity in the final results?

hard

A. Use MMR with a diversity weight of zero

B. Combine similarity and MMR scores with adjustable weights

C. Use only similarity scores to rank results

D. Randomly shuffle results after similarity ranking

Solution

1. Step 1: Understand the goal

   Balancing relevance and diversity requires combining both similarity and MMR scores meaningfully.

2. Step 2: Evaluate options

   Using only similarity or zero diversity weight ignores diversity; random shuffling loses relevance order.

3. Step 3: Best approach

   Combining similarity and MMR with adjustable weights allows tuning the balance effectively.

4. Final Answer:

   Combine similarity and MMR scores with adjustable weights -> Option B

5. Quick Check:

   Hybrid weighted combination = best balance [OK]

Hint: Adjust weights to balance relevance and diversity [OK]

Common Mistakes:

• Ignoring diversity by using similarity only
• Setting diversity weight to zero in MMR
• Randomizing results without scoring