TF-IDF and BM25 scoring in Elasticsearch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the BM25 score output for a simple match query?

Given an Elasticsearch index with documents containing a field text, what is the expected BM25 score output for the query {"match": {"text": "quick brown"}} on the document {"text": "the quick brown fox"}?

Assume default BM25 parameters and that the document is the only one in the index.

Elasticsearch

{
  "query": {
    "match": {
      "text": "quick brown"
    }
  }
}

A A positive float score close to 1.0

B Zero, because the query terms are stop words

C A negative score, since BM25 can produce negative values

D An error because BM25 does not support match queries

💡 Hint

BM25 scores documents based on term frequency and inverse document frequency. The score is always positive.

BM25 scoring produces a positive relevance score based on how well the document matches the query terms. Since the document contains both "quick" and "brown", the score will be positive and typically close to 1.0 for a single document.

🧠 Conceptual

intermediate

Which factor does TF-IDF consider that BM25 improves upon?

TF-IDF and BM25 are both scoring algorithms used in Elasticsearch. Which aspect does BM25 improve compared to classic TF-IDF?

A BM25 uses only exact phrase matches instead of individual terms

B BM25 removes inverse document frequency to simplify scoring

C BM25 ignores term frequency to speed up scoring

D BM25 adds document length normalization to better handle longer documents

💡 Hint

Think about how document length affects relevance scores.

BM25 improves upon TF-IDF by adding document length normalization, which prevents longer documents from unfairly scoring higher just because they have more terms.

🔧 Debug

advanced

Why does this Elasticsearch query return zero scores with BM25?

Consider this Elasticsearch query using BM25 scoring:

{
  "query": {
    "bool": {
      "must": [
        {"match": {"content": "apple orange"}}
      ]
    }
  }
}

The index contains documents with the field content, but all returned documents have a score of 0. What is the most likely cause?

A The field <code>content</code> is mapped as <code>keyword</code> type, so BM25 scoring does not apply

B BM25 scoring always returns zero for boolean queries

C The query syntax is invalid and causes scoring to reset to zero

D The documents do not contain the terms "apple" or "orange"

💡 Hint

Check the field mapping type for content.

BM25 scoring applies only to text fields analyzed with full-text analyzers. If content is mapped as keyword, it is not analyzed and BM25 scoring does not work, resulting in zero scores.

📝 Syntax

advanced

Which query correctly disables BM25 and uses classic TF-IDF in Elasticsearch?

Elasticsearch uses BM25 by default. Which query snippet correctly disables BM25 and enables classic TF-IDF scoring for the body field?

A { "match": { "body": { "query": "search text" } }, "similarity": { "type": "classic" } }

B { "match": { "body": { "query": "search text", "similarity": "classic" } } }

C { "match": { "body": "search text" }, "similarity": "classic" }

D { "match": { "body": { "query": "search text" } }, "similarity": "classic" }

💡 Hint

Similarity settings are usually configured at the field mapping or query level inside the field object.

Option B correctly sets the similarity parameter inside the match query for the body field, enabling classic TF-IDF scoring. Other options either place similarity outside the field or use invalid syntax.

🚀 Application

expert

How to tune BM25 parameters to favor term frequency over document length?

You want to adjust BM25 scoring in Elasticsearch to give more importance to how often a term appears in a document, and less importance to the document length. Which parameter settings achieve this?

A Set both <code>b</code> and <code>k1</code> to 0

B Set <code>b</code> close to 1 and <code>k1</code> to a lower value like 0.5

C Set <code>b</code> close to 0 and <code>k1</code> to a higher value like 2.0

D Set <code>b</code> to 0.75 and <code>k1</code> to 1.2 (default values)

💡 Hint

Remember b controls length normalization and k1 controls term frequency saturation.

Setting b close to 0 disables length normalization, so document length has less effect. Increasing k1 increases the impact of term frequency, making repeated terms more influential.

Practice

1. What is the main difference between TF-IDF and BM25 scoring in Elasticsearch?

easy

A. BM25 ignores inverse document frequency, TF-IDF uses it heavily.

B. TF-IDF is the default scoring algorithm in Elasticsearch, BM25 is optional.

C. BM25 considers term frequency saturation and document length normalization, TF-IDF does not.

D. TF-IDF scores are always higher than BM25 scores for the same query.

Solution

1. Step 1: Understand TF-IDF basics

   TF-IDF scores words by multiplying term frequency (TF) with inverse document frequency (IDF), but it does not adjust for document length or term frequency saturation.

2. Step 2: Understand BM25 improvements

   BM25 improves TF-IDF by adding term frequency saturation (diminishing returns for repeated terms) and normalizing by document length, making scores more balanced.

3. Final Answer:

   BM25 considers term frequency saturation and document length normalization, TF-IDF does not. -> Option C

4. Quick Check:

   BM25 adds saturation and length normalization [OK]

Hint: BM25 adjusts TF-IDF with saturation and length normalization [OK]

Common Mistakes:

• Thinking TF-IDF is default in Elasticsearch
• Believing BM25 ignores IDF
• Assuming TF-IDF always scores higher

2. Which Elasticsearch query parameter correctly switches the similarity scoring to TF-IDF instead of BM25?

easy

A. ""{
"similarity": "classic"
}"""

B. ""{
"similarity": "bm25"
}"""

C. ""{
"similarity": "tfidf"
}"""

D. ""{
"similarity": "default"
}"""

Solution

1. Step 1: Recall Elasticsearch similarity options

   Elasticsearch uses "bm25" by default. To switch to TF-IDF, you use the "classic" similarity setting.

2. Step 2: Match parameter to TF-IDF

   The "classic" similarity corresponds to TF-IDF scoring, so setting "similarity": "classic" enables TF-IDF.

3. Final Answer:

   { "similarity": "classic" } -> Option A

4. Quick Check:

   TF-IDF similarity is "classic" [OK]

Hint: Use similarity: classic to enable TF-IDF scoring [OK]

Common Mistakes:

• Using "tfidf" as similarity name (not valid)
• Assuming "default" switches to TF-IDF
• Confusing bm25 with TF-IDF

3. Given the following Elasticsearch query using BM25, what will happen to the score if a term appears multiple times in a document?

medium

A. The score increases but with diminishing returns due to term frequency saturation.

B. The score increases linearly with each additional term occurrence.

C. The score stays the same regardless of term frequency.

D. The score decreases as term frequency increases.

Solution

1. Step 1: Understand BM25 term frequency effect

   BM25 applies a saturation function to term frequency, so each additional occurrence adds less to the score than the previous one.

2. Step 2: Compare to linear increase

   Unlike TF-IDF which can increase linearly, BM25 limits the impact of repeated terms to avoid overvaluing them.

3. Final Answer:

   The score increases but with diminishing returns due to term frequency saturation. -> Option A

4. Quick Check:

   BM25 uses saturation, not linear increase [OK]

Hint: BM25 scores repeated terms with diminishing returns [OK]

Common Mistakes:

• Assuming linear score increase with term frequency
• Thinking score stays constant with repeats
• Believing score decreases with more terms

4. You configured Elasticsearch to use TF-IDF similarity but notice search results are not ranking as expected. Which of the following is a likely cause?

medium

A. TF-IDF requires a special query syntax different from standard queries.

B. TF-IDF does not use inverse document frequency, so scores are random.

C. BM25 must be disabled in the cluster settings to use TF-IDF.

D. You forgot to reindex the data after changing similarity settings.

Solution

1. Step 1: Understand similarity changes require reindexing

   Changing similarity settings affects how documents are scored, but Elasticsearch applies these only after reindexing the data.

2. Step 2: Identify common mistake

   If you don't reindex, old scores remain, so results won't reflect the new TF-IDF similarity.

3. Final Answer:

   You forgot to reindex the data after changing similarity settings. -> Option D

4. Quick Check:

   Reindexing needed after similarity change [OK]

Hint: Always reindex after changing similarity settings [OK]

Common Mistakes:

• Believing TF-IDF ignores IDF
• Thinking BM25 must be disabled cluster-wide
• Assuming query syntax changes for TF-IDF

5. You want to improve search relevance for a collection of very short documents using Elasticsearch. Which scoring approach and adjustment is best to handle short documents effectively?

hard

A. Use BM25 with a higher k1 parameter to ignore term frequency saturation.

B. Use BM25 with a lower b parameter to reduce length normalization effect.

C. Use TF-IDF similarity because it handles short documents better by default.

D. Use TF-IDF and disable inverse document frequency to boost short docs.

Solution

1. Step 1: Understand BM25 parameters for short docs

   BM25's b parameter controls length normalization. Lowering b reduces penalty on short documents, improving their scores.

2. Step 2: Compare to TF-IDF and other options

   TF-IDF lacks length normalization control. Increasing k1 ignores saturation, which can hurt relevance. Disabling IDF is incorrect.

3. Final Answer:

   Use BM25 with a lower b parameter to reduce length normalization effect. -> Option B

4. Quick Check:

   Lower b helps short docs in BM25 [OK]

Hint: Lower BM25 b parameter to favor short documents [OK]

Common Mistakes:

• Assuming TF-IDF is better for short docs by default
• Increasing k1 to ignore saturation hurts relevance
• Disabling IDF reduces search quality