Filter aggregation in Elasticsearch

Introduction

Filter aggregation helps you count or analyze only the data that matches certain rules. It lets you focus on a specific group inside your big data.

You want to count how many documents have a specific value in a field.

You need to find the average price of products only in a certain category.

You want to see how many users are from a particular country.

You want to analyze logs but only for errors with a certain status code.

Syntax

Elasticsearch

{
  "aggs": {
    "NAME": {
      "filter": {
        "TERM_OR_RANGE_QUERY"
      },
      "aggs": {
        "SUB_AGGREGATION": { ... }
      }
    }
  }
}

Replace NAME with a name you choose for this aggregation.

The filter part defines the rule to select documents.

Examples

This counts documents where the status field is exactly "error".

Elasticsearch

{
  "aggs": {
    "only_errors": {
      "filter": { "term": { "status": "error" } }
    }
  }
}

This filters products cheaper than 20 and calculates their average price.

Elasticsearch

{
  "aggs": {
    "cheap_products": {
      "filter": { "range": { "price": { "lt": 20 } } },
      "aggs": {
        "avg_price": { "avg": { "field": "price" } }
      }
    }
  }
}

Sample Program

This query finds users with status "active" and calculates their average age.

Elasticsearch

{
  "size": 0,
  "aggs": {
    "active_users": {
      "filter": {
        "term": { "status": "active" }
      },
      "aggs": {
        "average_age": {
          "avg": { "field": "age" }
        }
      }
    }
  }
}

Important Notes

The filter aggregation only counts documents matching the filter.

You can add sub-aggregations inside to calculate sums, averages, or other stats on the filtered data.

Summary

Filter aggregation selects documents matching a rule.

It helps analyze only a part of your data.

You can combine it with other aggregations for detailed insights.

Practice

1. What is the main purpose of a filter aggregation in Elasticsearch?

easy

A. To delete documents matching a query

B. To sort documents by a field value

C. To update documents in the index

D. To select documents that match a specific condition

Solution

1. Step 1: Understand filter aggregation role

   A filter aggregation is used to select documents that meet a certain condition or criteria.

2. Step 2: Compare with other options

   Sorting, updating, or deleting documents are not functions of filter aggregation.

3. Final Answer:

   To select documents that match a specific condition -> Option D

4. Quick Check:

   Filter aggregation = select matching documents [OK]

Hint: Filter aggregation selects matching documents only [OK]

Common Mistakes:

• Confusing filter aggregation with sorting
• Thinking filter aggregation modifies documents
• Mixing filter aggregation with delete operations

2. Which of the following is the correct syntax to define a filter aggregation in an Elasticsearch query?

easy

A. "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } }

B. "aggs": { "my_filter": { "terms": { "field": "status" } } }

C. "aggs": { "my_filter": { "range": { "field": "age", "gte": 30 } } }

D. "aggs": { "my_filter": { "match": { "status": "active" } } }

Solution

1. Step 1: Identify filter aggregation syntax

   The filter aggregation uses the key "filter" with a query inside, like "term" for exact match.

2. Step 2: Check each option

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } correctly uses "filter" with a "term" query. Options A, C, and D use other aggregation types or query types incorrectly.

3. Final Answer:

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } -> Option A

4. Quick Check:

   Filter aggregation syntax = option with "filter" key [OK]

Hint: Filter aggregation always uses "filter" key inside aggs [OK]

Common Mistakes:

• Using "terms" instead of "filter" for filter aggregation
• Confusing query types inside aggregation
• Missing the "filter" key in aggregation

3. Given this Elasticsearch aggregation query snippet, what will be the count of documents in the filter aggregation named active_users?

{
  "aggs": {
    "active_users": {
      "filter": { "term": { "status": "active" } }
    }
  }
}

Assuming the index has 1000 documents, 300 of which have status set to active.

medium

A. 300

B. 1000

C. 700

D. 0

Solution

1. Step 1: Understand filter aggregation result

   The filter aggregation counts documents matching the filter condition, here "status" = "active".

2. Step 2: Apply given data

   Out of 1000 documents, 300 have status "active", so the aggregation count is 300.

3. Final Answer:

   300 -> Option A

4. Quick Check:

   Filter count = matching docs = 300 [OK]

Hint: Filter aggregation count = number of matching documents [OK]

Common Mistakes:

• Choosing total documents instead of filtered count
• Confusing filter aggregation with terms aggregation count
• Assuming zero if filter is not exact

4. Identify the error in this filter aggregation query:

{
  "aggs": {
    "filtered_data": {
      "filter": {
        "terms": {
          "status": "active"
        }
      }
    }
  }
}

medium

A. The filter aggregation must use "term" not "terms" for single value

B. The "terms" query requires an array of values, not a single string

C. The aggregation name "filtered_data" is invalid

D. The filter aggregation cannot be nested inside "aggs"

Solution

1. Step 1: Check the "terms" query usage

   The "terms" query expects an array of values, e.g., ["active"], not a single string.

2. Step 2: Validate other parts

   The aggregation name is valid, and filter aggregation can be inside "aggs". Using "term" is an alternative but not an error here.

3. Final Answer:

   The "terms" query requires an array of values, not a single string -> Option B

4. Quick Check:

   "terms" query needs array, not string [OK]

Hint: "terms" query needs array of values, not single string [OK]

Common Mistakes:

• Using string instead of array for "terms" query
• Thinking aggregation names have restrictions
• Believing filter aggregation can't be nested

5. You want to find the average price of products only in the category "electronics" using filter aggregation. Which of the following query snippets correctly combines filter aggregation with an average sub-aggregation?

hard

A. {
"query": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}

B. {
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"filter": { "term": { "category": "electronics" } }
}
}

C. {
"aggs": {
"electronics_filter": {
"filter": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

D. {
"aggs": {
"electronics_filter": {
"terms": { "field": "category" },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

Solution

1. Step 1: Understand filter aggregation with sub-aggregation

   Filter aggregation uses "filter" key and can contain sub-aggregations inside "aggs".

2. Step 2: Check each option

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } correctly nests an average aggregation inside a filter aggregation for category "electronics". { "aggs": { "avg_price": { "avg": { "field": "price" } }, "filter": { "term": { "category": "electronics" } } } } misplaces filter outside aggregation. { "query": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } uses query filtering, not aggregation filtering. { "aggs": { "electronics_filter": { "terms": { "field": "category" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } uses "terms" aggregation, not filter.

3. Final Answer:

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } -> Option C

4. Quick Check:

   Filter aggregation with sub-aggregation = { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } [OK]

Hint: Nest avg aggregation inside filter aggregation's "aggs" [OK]

Common Mistakes:

• Placing filter outside aggregation block
• Using query filter instead of filter aggregation
• Confusing terms aggregation with filter aggregation