Nested aggregations in Elasticsearch - Step-by-Step Execution

Concept Flow - Nested aggregations

Start Query

↓

Outer Aggregation

↓

Inner Aggregation

↓

Collect Results

↓

Return Nested Aggregation Data

The query starts with an outer aggregation, then runs inner aggregations inside it, collects results, and returns nested aggregation data.

Execution Sample

Elasticsearch

{
  "aggs": {
    "by_category": {
      "terms": { "field": "category" },
      "aggs": {
        "avg_price": { "avg": { "field": "price" } }
      }
    }
  }
}

This query groups documents by category and calculates the average price within each category.

Execution Table

Step	Aggregation Level	Aggregation Type	Field	Action	Result
1	Outer	terms	category	Group documents by category field	Buckets created for each category
2	Inner	avg	price	Calculate average price in each category bucket	Average price computed per bucket
3	Collect	N/A	N/A	Collect all buckets with their average price	Nested aggregation result ready
4	Return	N/A	N/A	Return nested aggregation data to client	Final JSON aggregation response

💡 All buckets processed and nested aggregations computed, query execution complete

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
buckets	empty	buckets by category created	avg price added to each bucket	all buckets collected	nested aggregation result

Key Moments - 2 Insights

Why do we need nested aggregations instead of just one aggregation?

Nested aggregations let us group data first (outer aggregation) and then calculate metrics inside each group (inner aggregation), as shown in execution_table rows 1 and 2.

What happens if the inner aggregation is missing?

If inner aggregation is missing, only the outer aggregation groups data but no metrics like average are calculated inside groups, so step 2 in execution_table would be skipped.

Visual Quiz - 3 Questions

Look at the execution_table, what aggregation type is used at the outer level?

A avg

B terms

C sum

D max

💡 Hint

Check execution_table row 1 under Aggregation Type column

At which step is the average price calculated inside each category?

A Step 2

B Step 1

C Step 3

D Step 4

💡 Hint

Look at execution_table row 2 Action and Result columns

If we remove the inner aggregation, what will be missing in the results?

A Buckets grouped by category

B Final JSON response

C Average price per category

D Query execution

💡 Hint

Refer to key_moments explanation about missing inner aggregation and execution_table row 2

Concept Snapshot

Nested aggregations group data first, then calculate metrics inside groups.
Syntax: outer aggregation with 'aggs' containing inner aggregations.
Outer example: 'terms' aggregation.
Inner example: 'avg' aggregation.
Results return nested buckets with metrics.
Use to analyze grouped data with detailed stats.

Full Transcript

Nested aggregations in Elasticsearch start by grouping documents using an outer aggregation, such as 'terms' on a field like category. Then, inside each group, inner aggregations compute metrics like average price. The process collects all buckets with their computed metrics and returns a nested JSON response. This allows detailed analysis by grouping data first and then calculating statistics inside each group. The execution table shows steps from grouping to metric calculation to result collection and return. Variables like buckets change from empty to filled with grouped data and metrics. Key moments clarify why nested aggregations are needed and what happens if inner aggregations are missing. The visual quiz tests understanding of aggregation types, steps, and effects of removing inner aggregations.

Practice

1. What is the main purpose of nested aggregations in Elasticsearch?

easy

A. To sort documents by a single field

B. To group data inside other groups for multi-level analysis

C. To filter documents based on a condition

D. To update documents in bulk

Solution

1. Step 1: Recall aggregation basics

   Aggregations group data to summarize or analyze it.

2. Step 2: Examine nested aggregations

   Nested aggregations group data inside other groups for detailed multi-level insights.

3. Final Answer:

   To group data inside other groups for multi-level analysis -> Option B

4. Quick Check:

   Nested aggregations = multi-level grouping [OK]

Hint: Nested aggs always use 'aggs' inside another aggregation [OK]

Common Mistakes:

• Confusing nested aggregations with filters
• Thinking nested aggregations sort data
• Assuming nested aggregations update documents

2. Which of the following is the correct syntax to add a nested aggregation inside a terms aggregation in Elasticsearch?

easy

A. "aggs": { "country": { "terms": { "field": "country" }, "aggs": { "city": { "terms": { "field": "city" } } } } }

B. "nested": { "aggs": { "terms": { "field": "city" } } }

C. "aggs": { "nested_agg": { "terms": { "field": "city" } } }

D. "aggs": { "terms": { "field": "city" }, "aggs": { "nested_agg": { "terms": { "field": "country" } } } }

Solution

1. Step 1: Identify correct nested aggregation structure

   Nested aggregations are placed inside the "aggs" key of a parent aggregation.

2. Step 2: Check "aggs": { "country": { "terms": { "field": "country" }, "aggs": { "city": { "terms": { "field": "city" } } } } } structure

   "aggs": { "country": { "terms": { "field": "country" }, "aggs": { "city": { "terms": { "field": "city" } } } } } shows a terms aggregation on "country" with a nested terms aggregation on "city" inside its "aggs" key, which is correct syntax.

3. Final Answer:

   "aggs": { "country": { "terms": { "field": "country" }, "aggs": { "city": { "terms": { "field": "city" } } } } } -> Option A

4. Quick Check:

   Nested aggs inside parent's "aggs" key = "aggs": { "country": { "terms": { "field": "country" }, "aggs": { "city": { "terms": { "field": "city" } } } } } [OK]

Hint: Nested aggs go inside parent's "aggs" key, not outside [OK]

Common Mistakes:

• Placing multiple "aggs" keys at the same level
• Using "nested" keyword incorrectly
• Missing the "aggs" key inside parent aggregation

3. Given this Elasticsearch aggregation query snippet:

{
  "aggs": {
    "by_country": {
      "terms": { "field": "country" },
      "aggs": {
        "by_city": {
          "terms": { "field": "city" }
        }
      }
    }
  }
}

What does the nested aggregation by_city do?

medium

A. Groups documents by city inside each country group

B. Filters documents only from one city

C. Sorts countries by city name

D. Counts total documents ignoring country

Solution

1. Step 1: Analyze the outer aggregation

   "by_country" groups documents by the "country" field.

2. Step 2: Analyze the nested aggregation

   Inside each country group, "by_city" groups documents by the "city" field.

3. Final Answer:

   Groups documents by city inside each country group -> Option A

4. Quick Check:

   Nested terms aggregation groups inside parent group [OK]

Hint: Nested terms aggs group data inside parent buckets [OK]

Common Mistakes:

• Thinking nested aggs filter data
• Assuming nested aggs sort data
• Ignoring the parent aggregation grouping

4. Identify the error in this nested aggregation query:

{
  "aggs": {
    "countries": {
      "terms": { "field": "country" },
      "terms": {
        "field": "city"
      }
    }
  }
}

medium

A. Using "terms" twice at the same level is correct

B. Aggregation names cannot be plural

C. Field names "country" and "city" must be the same

D. Missing nested "aggs" key for the city aggregation

Solution

1. Step 1: Check aggregation structure

   Each aggregation can have only one aggregation type key like "terms" at its level.

2. Step 2: Identify nested aggregation placement

   Nested aggregations must be inside an "aggs" key, but here "terms" is repeated instead of nesting inside "aggs".

3. Final Answer:

   Missing nested "aggs" key for the city aggregation -> Option D

4. Quick Check:

   Nested aggs require "aggs" key inside parent [OK]

Hint: Nested aggs must be inside "aggs", not repeated aggregation keys [OK]

Common Mistakes:

• Repeating aggregation type keys at same level
• Not nesting inside "aggs" key
• Confusing aggregation names with field names

5. You want to analyze sales data by region, then by product category, and finally by brand using nested aggregations. Which of the following query structures correctly implements this three-level nested aggregation?

hard

A. { "aggs": { "region": { "terms": { "field": "region" }, "terms": { "field": "category" }, "terms": { "field": "brand" } } } }

B. { "aggs": { "region": { "terms": { "field": "region" } }, "category": { "terms": { "field": "category" } }, "brand": { "terms": { "field": "brand" } } } }

C. { "aggs": { "region": { "terms": { "field": "region" }, "aggs": { "category": { "terms": { "field": "category" }, "aggs": { "brand": { "terms": { "field": "brand" } } } } } } } }

D. { "aggs": { "brand": { "terms": { "field": "brand" }, "aggs": { "category": { "terms": { "field": "category" }, "aggs": { "region": { "terms": { "field": "region" } } } } } } } }

Solution

1. Step 1: Identify the required nesting order

   We want to group first by region, then inside each region by category, then inside each category by brand.

2. Step 2: Check each option's structure

   { "aggs": { "region": { "terms": { "field": "region" }, "aggs": { "category": { "terms": { "field": "category" }, "aggs": { "brand": { "terms": { "field": "brand" } } } } } } } correctly nests "category" inside "region"'s "aggs", and "brand" inside "category"'s "aggs". Others either flatten or reverse order.

3. Final Answer:

   { "aggs": { "region": { "terms": { "field": "region" }, "aggs": { "category": { "terms": { "field": "category" }, "aggs": { "brand": { "terms": { "field": "brand" } } } } } } } -> Option C

4. Quick Check:

   Nested aggs inside parent's "aggs" in correct order = { "aggs": { "region": { "terms": { "field": "region" }, "aggs": { "category": { "terms": { "field": "category" }, "aggs": { "brand": { "terms": { "field": "brand" } } } } } } } [OK]

Hint: Nest each level inside previous level's "aggs" key [OK]

Common Mistakes:

• Placing all aggregations at top level without nesting
• Repeating aggregation keys at same level
• Reversing the desired grouping order