Bool query in depth in Elasticsearch - Step-by-Step Execution

Concept Flow - Bool query in depth

Start Bool Query

↓

Evaluate must clauses

↓

Evaluate filter clauses

↓

Evaluate should clauses

↓

Evaluate must_not clauses

↓

Combine results

↓

Return matching documents

The Bool query combines multiple queries using must, filter, should, and must_not clauses to find matching documents.

Execution Sample

Elasticsearch

{
  "bool": {
    "must": [{ "match": { "title": "apple" }}],
    "filter": [{ "term": { "status": "published" }}],
    "should": [{ "match": { "description": "fresh" }}],
    "must_not": [{ "term": { "category": "expired" }}]
  }
}

This Bool query finds documents with title 'apple', status 'published', optionally matching 'fresh' in description, and excludes category 'expired'.

Execution Table

Step	Clause Type	Clause Query	Evaluation Result	Effect on Result
1	must	match title: apple	Documents with 'apple' in title	Include only these documents
2	filter	term status: published	Documents with status 'published'	Filter documents to these only
3	should	match description: fresh	Documents with 'fresh' in description	Boost score if matched, but not required
4	must_not	term category: expired	Documents with category 'expired'	Exclude these documents
5	combine	All clauses combined	Documents matching must & filter, excluding must_not, optionally boosted by should	Final result set returned

💡 All clauses evaluated and combined to produce final matching documents

Variable Tracker

Variable	Start	After must	After filter	After should	After must_not	Final
Matching Documents	All documents	Docs with title 'apple'	Docs with title 'apple' and status 'published'	Docs with title 'apple' and status 'published' boosted if description has 'fresh'	Docs excluding category 'expired'	Final filtered and scored documents

Key Moments - 3 Insights

Why does the filter clause not affect scoring?

Filter clauses only include or exclude documents without changing their relevance score, as shown in step 2 of the execution_table.

What happens if no should clauses match?

If must clauses exist, should clauses only boost scores but are not required; documents still match if must and filter clauses pass, as seen in step 3.

How does must_not affect the result?

Documents matching must_not clauses are excluded entirely from results, as shown in step 4.

Visual Quiz - 3 Questions

Look at the execution_table, what is the effect of the must clause at step 1?

A Boost score of documents with 'apple' in title

B Exclude documents with 'apple' in title

C Include only documents with 'apple' in title

D Filter documents with status 'published'

💡 Hint

Check step 1 under 'Effect on Result' in execution_table

At which step are documents excluded if they have category 'expired'?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Look for must_not clause evaluation in execution_table

If the filter clause was removed, how would the variable 'Matching Documents' change after step 2?

A It would be documents with title 'apple' only

B It would remain all documents

C It would exclude documents with category 'expired'

D It would boost documents with 'fresh' in description

💡 Hint

Refer to variable_tracker and execution_table step 2 for filter clause effect

Concept Snapshot

Bool query syntax:
{
  "bool": {
    "must": [...],
    "filter": [...],
    "should": [...],
    "must_not": [...]
  }
}

Behavior:
must = required clauses,
filter = required but no scoring,
should = optional, boosts score,
must_not = excludes matches.

Key rule: All must and filter clauses must match; must_not excludes; should boosts scoring.

Full Transcript

The Bool query in Elasticsearch combines multiple queries using must, filter, should, and must_not clauses. First, it evaluates must clauses to include documents that match required conditions. Then, filter clauses further restrict documents without affecting their score. Should clauses optionally boost the score of documents if matched but are not required if must clauses exist. Must_not clauses exclude documents that match their conditions. Finally, all results are combined to return documents that satisfy must and filter clauses, exclude must_not matches, and are optionally boosted by should matches.

Practice

1. Which clause in an Elasticsearch bool query is used to exclude documents from the search results?

easy

A. must

B. should

C. must_not

D. filter

Solution

1. Step 1: Understand bool query clauses

   The bool query has four main clauses: must (required matches), filter (fast filtering), should (optional matches), and must_not (exclusions).

2. Step 2: Identify exclusion clause

   The must_not clause excludes documents that match its conditions from the results.

3. Final Answer:

   must_not -> Option C

4. Quick Check:

   Exclusion = must_not [OK]

Hint: Exclude with must_not clause in bool query [OK]

Common Mistakes:

• Confusing must_not with should
• Using filter to exclude documents
• Thinking must excludes documents

2. Which of the following is the correct syntax to add a filter clause inside a bool query in Elasticsearch?

easy

A. "bool": { "must": [ { "term": { "status": "active" } } ] }

B. "bool": { "must_not": { "term": { "status": "active" } } }

C. "bool": { "should": { "term": { "status": "active" } } }

D. "bool": { "filter": [ { "term": { "status": "active" } } ] }

Solution

1. Step 1: Recall bool query syntax for filter

   The filter clause expects an array of queries inside the bool query, e.g., "filter": [ { ... } ].

2. Step 2: Match correct syntax

   "bool": { "filter": [ { "term": { "status": "active" } } ] } correctly uses "filter": [ { "term": { "status": "active" } } ] which is valid syntax.

3. Final Answer:

   "bool": { "filter": [ { "term": { "status": "active" } } ] } -> Option D

4. Quick Check:

   Filter clause uses array syntax [OK]

Hint: Filter clause always uses an array of queries [OK]

Common Mistakes:

• Using object instead of array for filter
• Confusing must_not or should syntax
• Missing brackets around filter queries

3. Given this bool query snippet, what documents will it match?

{ "bool": { "must": [ { "match": { "title": "python" } } ], "should": [ { "term": { "tags": "programming" } } ], "must_not": [ { "term": { "status": "draft" } } ] } }

medium

A. Documents with title containing 'python', optionally tagged 'programming', excluding drafts

B. Documents tagged 'programming' only, ignoring title and status

C. Documents with status 'draft' and title 'python'

D. Documents with title 'python' or status 'draft'

Solution

1. Step 1: Analyze must clause

   The must clause requires documents to have 'python' in the title.

2. Step 2: Analyze should and must_not clauses

   The should clause boosts documents tagged 'programming' but is optional. The must_not clause excludes documents with status 'draft'.

3. Final Answer:

   Documents with title containing 'python', optionally tagged 'programming', excluding drafts -> Option A

4. Quick Check:

   Must + should + must_not = Documents with title containing 'python', optionally tagged 'programming', excluding drafts [OK]

Hint: Must is required, should is optional, must_not excludes [OK]

Common Mistakes:

• Thinking should is required
• Including documents with status 'draft'
• Ignoring must clause

4. Identify the error in this bool query snippet:

{ "bool": { "must": { "term": { "user": "kimchy" } }, "filter": { "term": { "status": "active" } } } }

medium

A. Missing "should" clause

B. "must" and "filter" should be arrays, not single objects

C. Using "term" inside bool is invalid

D. The query is correct as is

Solution

1. Step 1: Check bool clause types

   In bool queries, must and filter expect arrays of queries, not single objects.

2. Step 2: Identify syntax error

   Here, both must and filter are single objects, which is invalid syntax.

3. Final Answer:

   "must" and "filter" should be arrays, not single objects -> Option B

4. Quick Check:

   Bool clauses require arrays [OK]

Hint: Bool clauses must and filter need arrays [OK]

Common Mistakes:

• Using single object instead of array
• Forgetting to wrap queries in []
• Assuming should is mandatory

5. You want to find documents where the category is 'books' and either the author is 'Alice' or the year is after 2015, but exclude any documents marked as archived. Which bool query structure correctly implements this?

hard

A. { "bool": { "must": [ { "term": { "category": "books" } } ], "should": [ { "term": { "author": "Alice" } }, { "range": { "year": { "gt": 2015 } } } ], "must_not": [ { "term": { "status": "archived" } } ] } }

B. { "bool": { "filter": [ { "term": { "category": "books" } }, { "term": { "author": "Alice" } } ], "must_not": [ { "term": { "status": "archived" } } ], "should": [ { "range": { "year": { "gt": 2015 } } } ] } }

C. { "bool": { "must": [ { "term": { "category": "books" } }, { "term": { "author": "Alice" } } ], "must_not": [ { "term": { "status": "archived" } } ], "should": [ { "range": { "year": { "gt": 2015 } } } ] } }

D. { "bool": { "must": [ { "term": { "category": "books" } } ], "filter": [ { "term": { "author": "Alice" } }, { "range": { "year": { "gt": 2015 } } } ], "must_not": [ { "term": { "status": "archived" } } ] } }

Solution

1. Step 1: Identify required and optional conditions

   The category must be 'books' (required), and either author is 'Alice' or year > 2015 (optional, so use should).

2. Step 2: Exclude archived documents

   Use must_not to exclude documents with status 'archived'.

3. Step 3: Match query structure

   { "bool": { "must": [ { "term": { "category": "books" } } ], "should": [ { "term": { "author": "Alice" } }, { "range": { "year": { "gt": 2015 } } } ], "must_not": [ { "term": { "status": "archived" } } ] } } correctly uses must for category, should for author or year, and must_not for exclusion.

4. Final Answer:

   Option A's bool query structure -> Option A

5. Quick Check:

   Must + should + must_not for conditions [OK]

Hint: Use must for required, should for alternatives, must_not to exclude [OK]

Common Mistakes:

• Putting author and year in must (both required)
• Using filter instead of should for alternatives
• Missing must_not for exclusion