Filter aggregation in Elasticsearch - Step-by-Step Execution

Concept Flow - Filter aggregation

Start Query

↓

Apply Filter Condition

↓

Select Documents Matching Filter

↓

Perform Aggregations on Filtered Docs

↓

Return Aggregation Results

The filter aggregation applies a filter to select documents, then runs aggregations only on those filtered documents.

Execution Sample

Elasticsearch

{
  "aggs": {
    "filtered_sales": {
      "filter": { "term": { "status": "sold" } },
      "aggs": { "total_price": { "sum": { "field": "price" } } }
    }
  }
}

This query filters documents where status is 'sold' and sums the price field for those documents.

Execution Table

Step	Action	Filter Condition	Documents Selected	Aggregation Performed	Result
1	Start query	N/A	All documents	N/A	N/A
2	Apply filter	status == 'sold'	Only documents with status 'sold'	N/A	N/A
3	Run aggregation	N/A	Filtered documents	Sum price field	Sum of prices for sold items
4	Return result	N/A	N/A	N/A	{"filtered_sales":{"doc_count":3,"total_price":{"value":450}}}

💡 Aggregation completes after filtering and summing prices of matching documents.

Variable Tracker

Variable	Start	After Filter	After Aggregation	Final
Documents	All documents	Filtered to status='sold'	Aggregation on filtered docs	Aggregation result returned
doc_count	N/A	3 (example count)	N/A	3
total_price	N/A	N/A	450 (example sum)	450

Key Moments - 2 Insights

Why does the aggregation only include some documents?

Because the filter aggregation first selects documents matching the filter condition (see execution_table step 2), only those documents are used in the aggregation.

What happens if no documents match the filter?

The aggregation runs on zero documents, so the result will show doc_count as 0 and aggregation values as zero or null (see variable_tracker doc_count after filter).

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what documents are selected?

A All documents in the index

B Documents where price is zero

C Documents where status is 'sold'

D No documents

💡 Hint

Check the 'Documents Selected' column at step 2 in execution_table.

At which step is the sum of the price field calculated?

A Step 2

B Step 3

C Step 1

D Step 4

💡 Hint

Look at the 'Aggregation Performed' column in execution_table.

If the filter changed to status='available', how would the doc_count change?

A It would be the count of documents with status 'available'

B It would stay the same

C It would be zero always

D It would count all documents

💡 Hint

Refer to variable_tracker doc_count after filter and how filter affects document selection.

Concept Snapshot

Filter aggregation syntax:
{
  "aggs": {
    "name": {
      "filter": { <filter_query> },
      "aggs": { <sub_aggregations> }
    }
  }
}

Behavior: Filters documents first, then runs sub-aggregations only on those.
Key rule: Aggregations inside filter only see filtered docs.

Full Transcript

Filter aggregation in Elasticsearch works by first applying a filter condition to select only documents that match. Then, it performs aggregations like sum or average only on those filtered documents. For example, filtering documents where status is 'sold' and summing their price fields. The execution steps are: start query, apply filter, run aggregation on filtered docs, and return results. Variables like doc_count track how many documents matched the filter. If no documents match, aggregation results reflect zero or null values. Changing the filter changes which documents are included and thus changes aggregation results.

Practice

1. What is the main purpose of a filter aggregation in Elasticsearch?

easy

A. To delete documents matching a query

B. To sort documents by a field value

C. To update documents in the index

D. To select documents that match a specific condition

Solution

1. Step 1: Understand filter aggregation role

   A filter aggregation is used to select documents that meet a certain condition or criteria.

2. Step 2: Compare with other options

   Sorting, updating, or deleting documents are not functions of filter aggregation.

3. Final Answer:

   To select documents that match a specific condition -> Option D

4. Quick Check:

   Filter aggregation = select matching documents [OK]

Hint: Filter aggregation selects matching documents only [OK]

Common Mistakes:

• Confusing filter aggregation with sorting
• Thinking filter aggregation modifies documents
• Mixing filter aggregation with delete operations

2. Which of the following is the correct syntax to define a filter aggregation in an Elasticsearch query?

easy

A. "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } }

B. "aggs": { "my_filter": { "terms": { "field": "status" } } }

C. "aggs": { "my_filter": { "range": { "field": "age", "gte": 30 } } }

D. "aggs": { "my_filter": { "match": { "status": "active" } } }

Solution

1. Step 1: Identify filter aggregation syntax

   The filter aggregation uses the key "filter" with a query inside, like "term" for exact match.

2. Step 2: Check each option

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } correctly uses "filter" with a "term" query. Options A, C, and D use other aggregation types or query types incorrectly.

3. Final Answer:

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } -> Option A

4. Quick Check:

   Filter aggregation syntax = option with "filter" key [OK]

Hint: Filter aggregation always uses "filter" key inside aggs [OK]

Common Mistakes:

• Using "terms" instead of "filter" for filter aggregation
• Confusing query types inside aggregation
• Missing the "filter" key in aggregation

3. Given this Elasticsearch aggregation query snippet, what will be the count of documents in the filter aggregation named active_users?

{
  "aggs": {
    "active_users": {
      "filter": { "term": { "status": "active" } }
    }
  }
}

Assuming the index has 1000 documents, 300 of which have status set to active.

medium

A. 300

B. 1000

C. 700

D. 0

Solution

1. Step 1: Understand filter aggregation result

   The filter aggregation counts documents matching the filter condition, here "status" = "active".

2. Step 2: Apply given data

   Out of 1000 documents, 300 have status "active", so the aggregation count is 300.

3. Final Answer:

   300 -> Option A

4. Quick Check:

   Filter count = matching docs = 300 [OK]

Hint: Filter aggregation count = number of matching documents [OK]

Common Mistakes:

• Choosing total documents instead of filtered count
• Confusing filter aggregation with terms aggregation count
• Assuming zero if filter is not exact

4. Identify the error in this filter aggregation query:

{
  "aggs": {
    "filtered_data": {
      "filter": {
        "terms": {
          "status": "active"
        }
      }
    }
  }
}

medium

A. The filter aggregation must use "term" not "terms" for single value

B. The "terms" query requires an array of values, not a single string

C. The aggregation name "filtered_data" is invalid

D. The filter aggregation cannot be nested inside "aggs"

Solution

1. Step 1: Check the "terms" query usage

   The "terms" query expects an array of values, e.g., ["active"], not a single string.

2. Step 2: Validate other parts

   The aggregation name is valid, and filter aggregation can be inside "aggs". Using "term" is an alternative but not an error here.

3. Final Answer:

   The "terms" query requires an array of values, not a single string -> Option B

4. Quick Check:

   "terms" query needs array, not string [OK]

Hint: "terms" query needs array of values, not single string [OK]

Common Mistakes:

• Using string instead of array for "terms" query
• Thinking aggregation names have restrictions
• Believing filter aggregation can't be nested

5. You want to find the average price of products only in the category "electronics" using filter aggregation. Which of the following query snippets correctly combines filter aggregation with an average sub-aggregation?

hard

A. {
"query": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}

B. {
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"filter": { "term": { "category": "electronics" } }
}
}

C. {
"aggs": {
"electronics_filter": {
"filter": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

D. {
"aggs": {
"electronics_filter": {
"terms": { "field": "category" },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

Solution

1. Step 1: Understand filter aggregation with sub-aggregation

   Filter aggregation uses "filter" key and can contain sub-aggregations inside "aggs".

2. Step 2: Check each option

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } correctly nests an average aggregation inside a filter aggregation for category "electronics". { "aggs": { "avg_price": { "avg": { "field": "price" } }, "filter": { "term": { "category": "electronics" } } } } misplaces filter outside aggregation. { "query": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } uses query filtering, not aggregation filtering. { "aggs": { "electronics_filter": { "terms": { "field": "category" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } uses "terms" aggregation, not filter.

3. Final Answer:

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } -> Option C

4. Quick Check:

   Filter aggregation with sub-aggregation = { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } [OK]

Hint: Nest avg aggregation inside filter aggregation's "aggs" [OK]

Common Mistakes:

• Placing filter outside aggregation block
• Using query filter instead of filter aggregation
• Confusing terms aggregation with filter aggregation