Filter aggregation in Elasticsearch - Mini Project: Build & Apply

Filter aggregation

📖 Scenario: You have a collection of products in an online store. Each product has a category and a price. You want to find out how many products are in the electronics category and cost less than 500.

🎯 Goal: Create an Elasticsearch query using filter aggregation to count products that belong to the electronics category and have a price less than 500.

📋 What You'll Learn

Create a filter aggregation named electronics_under_500

Use a bool query with must conditions for category equals electronics and price less than 500

Include a match_all query at the top level

Print the full JSON query as the final output

💡 Why This Matters

🌍 Real World

Filter aggregations help online stores analyze product counts based on multiple conditions, like category and price, to make business decisions.

💼 Career

Understanding filter aggregations is important for roles involving Elasticsearch, data analysis, and search engine optimization.

1

Create the base query

Create a JSON object called query with a match_all query inside it.

Elasticsearch

{
  "query": {
    "match_all": {}
  }
}
# Your code here

Hint

The match_all query returns all documents.

2

Add filter aggregation

Add an aggs field to the JSON with a filter aggregation named electronics_under_500.

Elasticsearch

{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "electronics_under_500": {
      "filter": {
        
      }
    }
  }
}
# Your code here

Hint

The filter aggregation lets you count documents matching a filter.

3

Define the filter conditions

Inside the filter aggregation, add a bool query with must conditions: one for term category equals electronics and one for range price less than 500.

Elasticsearch

{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "electronics_under_500": {
      "filter": {
        "bool": {
          "must": [
            
          ]
        }
      }
    }
  }
}
# Your code here

Hint

Use bool with must array to combine filters.

4

Print the final query

Print the full JSON query object exactly as defined in the previous steps.

Elasticsearch

{
  "query": {
    "match_all": {}
  },
  "aggs": {
    "electronics_under_500": {
      "filter": {
        "bool": {
          "must": [
            { "term": { "category": "electronics" } },
            { "range": { "price": { "lt": 500 } } }
          ]
        }
      }
    }
  }
}
# Your code here

Hint

Use print() to display the JSON object.

Practice

1. What is the main purpose of a filter aggregation in Elasticsearch?

easy

A. To delete documents matching a query

B. To sort documents by a field value

C. To update documents in the index

D. To select documents that match a specific condition

Solution

1. Step 1: Understand filter aggregation role

   A filter aggregation is used to select documents that meet a certain condition or criteria.

2. Step 2: Compare with other options

   Sorting, updating, or deleting documents are not functions of filter aggregation.

3. Final Answer:

   To select documents that match a specific condition -> Option D

4. Quick Check:

   Filter aggregation = select matching documents [OK]

Hint: Filter aggregation selects matching documents only [OK]

Common Mistakes:

• Confusing filter aggregation with sorting
• Thinking filter aggregation modifies documents
• Mixing filter aggregation with delete operations

2. Which of the following is the correct syntax to define a filter aggregation in an Elasticsearch query?

easy

A. "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } }

B. "aggs": { "my_filter": { "terms": { "field": "status" } } }

C. "aggs": { "my_filter": { "range": { "field": "age", "gte": 30 } } }

D. "aggs": { "my_filter": { "match": { "status": "active" } } }

Solution

1. Step 1: Identify filter aggregation syntax

   The filter aggregation uses the key "filter" with a query inside, like "term" for exact match.

2. Step 2: Check each option

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } correctly uses "filter" with a "term" query. Options A, C, and D use other aggregation types or query types incorrectly.

3. Final Answer:

   "aggs": { "my_filter": { "filter": { "term": { "status": "active" } } } } -> Option A

4. Quick Check:

   Filter aggregation syntax = option with "filter" key [OK]

Hint: Filter aggregation always uses "filter" key inside aggs [OK]

Common Mistakes:

• Using "terms" instead of "filter" for filter aggregation
• Confusing query types inside aggregation
• Missing the "filter" key in aggregation

3. Given this Elasticsearch aggregation query snippet, what will be the count of documents in the filter aggregation named active_users?

{
  "aggs": {
    "active_users": {
      "filter": { "term": { "status": "active" } }
    }
  }
}

Assuming the index has 1000 documents, 300 of which have status set to active.

medium

A. 300

B. 1000

C. 700

D. 0

Solution

1. Step 1: Understand filter aggregation result

   The filter aggregation counts documents matching the filter condition, here "status" = "active".

2. Step 2: Apply given data

   Out of 1000 documents, 300 have status "active", so the aggregation count is 300.

3. Final Answer:

   300 -> Option A

4. Quick Check:

   Filter count = matching docs = 300 [OK]

Hint: Filter aggregation count = number of matching documents [OK]

Common Mistakes:

• Choosing total documents instead of filtered count
• Confusing filter aggregation with terms aggregation count
• Assuming zero if filter is not exact

4. Identify the error in this filter aggregation query:

{
  "aggs": {
    "filtered_data": {
      "filter": {
        "terms": {
          "status": "active"
        }
      }
    }
  }
}

medium

A. The filter aggregation must use "term" not "terms" for single value

B. The "terms" query requires an array of values, not a single string

C. The aggregation name "filtered_data" is invalid

D. The filter aggregation cannot be nested inside "aggs"

Solution

1. Step 1: Check the "terms" query usage

   The "terms" query expects an array of values, e.g., ["active"], not a single string.

2. Step 2: Validate other parts

   The aggregation name is valid, and filter aggregation can be inside "aggs". Using "term" is an alternative but not an error here.

3. Final Answer:

   The "terms" query requires an array of values, not a single string -> Option B

4. Quick Check:

   "terms" query needs array, not string [OK]

Hint: "terms" query needs array of values, not single string [OK]

Common Mistakes:

• Using string instead of array for "terms" query
• Thinking aggregation names have restrictions
• Believing filter aggregation can't be nested

5. You want to find the average price of products only in the category "electronics" using filter aggregation. Which of the following query snippets correctly combines filter aggregation with an average sub-aggregation?

hard

A. {
"query": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}

B. {
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"filter": { "term": { "category": "electronics" } }
}
}

C. {
"aggs": {
"electronics_filter": {
"filter": { "term": { "category": "electronics" } },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

D. {
"aggs": {
"electronics_filter": {
"terms": { "field": "category" },
"aggs": { "avg_price": { "avg": { "field": "price" } } }
}
}
}

Solution

1. Step 1: Understand filter aggregation with sub-aggregation

   Filter aggregation uses "filter" key and can contain sub-aggregations inside "aggs".

2. Step 2: Check each option

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } correctly nests an average aggregation inside a filter aggregation for category "electronics". { "aggs": { "avg_price": { "avg": { "field": "price" } }, "filter": { "term": { "category": "electronics" } } } } misplaces filter outside aggregation. { "query": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } uses query filtering, not aggregation filtering. { "aggs": { "electronics_filter": { "terms": { "field": "category" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } uses "terms" aggregation, not filter.

3. Final Answer:

   { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } -> Option C

4. Quick Check:

   Filter aggregation with sub-aggregation = { "aggs": { "electronics_filter": { "filter": { "term": { "category": "electronics" } }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } } [OK]

Hint: Nest avg aggregation inside filter aggregation's "aggs" [OK]

Common Mistakes:

• Placing filter outside aggregation block
• Using query filter instead of filter aggregation
• Confusing terms aggregation with filter aggregation