Stats and extended stats in Elasticsearch - Mini Project: Build & Apply

Stats and extended stats

📖 Scenario: You are working with a small dataset of sales amounts in an Elasticsearch index. You want to calculate basic statistics like count, min, max, and average, and then extend it to include sum of squares, variance, and standard deviation.

🎯 Goal: Build Elasticsearch aggregation queries step-by-step to calculate stats and extended_stats on the sales field.

📋 What You'll Learn

Create an aggregation query with stats on the sales field

Add a variable field_name with value sales

Create an aggregation query with extended_stats on the sales field using the variable

Print the final JSON aggregation query

💡 Why This Matters

🌍 Real World

Calculating statistics on sales data helps businesses understand performance, detect trends, and make decisions.

💼 Career

Elasticsearch is widely used for search and analytics. Knowing how to write aggregation queries for stats is valuable for data engineers and backend developers.

1

Create a stats aggregation query

Create a JSON object called stats_query with an aggregation named sales_stats that uses the stats aggregation on the field sales.

Elasticsearch

# Your code here

Hint

Use the stats aggregation inside aggs with the field sales.

2

Add a field name variable

Create a variable called field_name and set it to the string "sales".

Elasticsearch

stats_query = {
    "aggs": {
        "sales_stats": {
            "stats": {
                "field": "sales"
            }
        }
    }
}

# Your code here

Hint

Just assign the string "sales" to the variable field_name.

3

Create an extended_stats aggregation using the variable

Create a JSON object called extended_stats_query with an aggregation named sales_extended_stats that uses the extended_stats aggregation on the field stored in the variable field_name.

Elasticsearch

stats_query = {
    "aggs": {
        "sales_stats": {
            "stats": {
                "field": "sales"
            }
        }
    }
}

field_name = "sales"

# Your code here

Hint

Use the variable field_name as the value for the field key inside extended_stats.

4

Print the extended_stats query

Write a print statement to display the extended_stats_query JSON object.

Elasticsearch

stats_query = {
    "aggs": {
        "sales_stats": {
            "stats": {
                "field": "sales"
            }
        }
    }
}

field_name = "sales"

extended_stats_query = {
    "aggs": {
        "sales_extended_stats": {
            "extended_stats": {
                "field": field_name
            }
        }
    }
}

# Your code here

Hint

Use print(extended_stats_query) to show the final JSON.

Practice

1. What does the stats aggregation in Elasticsearch provide by default?

easy

A. Count, min, max, avg, and sum

B. Variance and standard deviation

C. Sum of squares only

D. Median and mode

Solution

1. Step 1: Understand the purpose of stats aggregation

   The stats aggregation provides basic statistical data about numeric fields.

2. Step 2: Identify the default metrics included

   By default, it returns count, min, max, avg, and sum values.

3. Final Answer:

   Count, min, max, avg, and sum -> Option A

4. Quick Check:

   Stats aggregation = basic stats [OK]

Hint: Stats aggregation gives basic stats like count and sum [OK]

Common Mistakes:

• Confusing stats with extended_stats
• Thinking stats includes variance
• Assuming median is included

2. Which of the following is the correct syntax to add an extended stats aggregation named my_stats on the field price?

easy

A. { "aggs": { "my_stats": { "extended_stats": { "field": "price" } } } }

B. { "aggs": { "my_stats": { "stats": { "field": "price" } } } }

C. { "aggs": { "my_stats": { "extended_stats": { "field_name": "price" } } } }

D. { "aggregations": { "my_stats": { "stats": { "field": "price" } } } }

Solution

1. Step 1: Check the aggregation type and field key

   The aggregation type must be extended_stats and the field key must be field.

2. Step 2: Verify the JSON structure

   The correct syntax uses aggs as the root key for aggregations, not aggregations.

3. Final Answer:

   { "aggs": { "my_stats": { "extended_stats": { "field": "price" } } } } -> Option A

4. Quick Check:

   Use extended_stats with field inside aggs [OK]

Hint: Use "aggs" and "extended_stats" with "field" key [OK]

Common Mistakes:

• Using "field_name" instead of "field"
• Using "aggregations" instead of "aggs"
• Confusing stats with extended_stats

3. Given this Elasticsearch aggregation query:

{
  "aggs": {
    "price_stats": {
      "extended_stats": {
        "field": "price"
      }
    }
  }
}

Which of the following metrics will NOT be included in the response?

medium

A. Sum of squares

B. Median

C. Standard deviation

D. Variance

Solution

1. Step 1: Identify metrics returned by extended_stats

   The extended_stats aggregation returns count, min, max, avg, sum, sum_of_squares, variance, and std_deviation.

2. Step 2: Check which metric is missing

   Median is not included in either stats or extended_stats aggregations.

3. Final Answer:

   Median -> Option B

4. Quick Check:

   Median not in extended_stats [OK]

Hint: Extended stats exclude median, include variance and std deviation [OK]

Common Mistakes:

• Assuming median is part of extended_stats
• Confusing sum_of_squares with sum
• Thinking variance is not included

4. You wrote this aggregation:

{
  "aggs": {
    "my_stats": {
      "extended_stats": {
        "field_name": "age"
      }
    }
  }
}

Why does this query fail?

medium

A. The aggregation name cannot be "my_stats"

B. The root key should be "aggregations" instead of "aggs"

C. extended_stats does not support the field "age"

D. The field key should be "field", not "field_name"

Solution

1. Step 1: Check the field key in extended_stats

   The correct key to specify the field is field, not field_name.

2. Step 2: Verify other parts of the query

   The aggregation name "my_stats" is valid, and "aggs" is the correct root key. The field "age" can be used if it exists.

3. Final Answer:

   The field key should be "field", not "field_name" -> Option D

4. Quick Check:

   Use "field" key for field name [OK]

Hint: Use "field" key, not "field_name" in aggregations [OK]

Common Mistakes:

• Using "field_name" instead of "field"
• Changing aggregation root key incorrectly
• Assuming field name is invalid without checking

5. You want to find the average price and also understand the price spread using variance and standard deviation in one query. Which aggregation setup is best?

hard

A. Use stats aggregation on price

B. Use percentiles aggregation on price

C. Use extended_stats aggregation on price

D. Use terms aggregation on price

Solution

1. Step 1: Identify metrics needed

   You want average, variance, and standard deviation, which describe spread and central tendency.

2. Step 2: Choose aggregation that provides all metrics

   extended_stats provides avg, variance, std_deviation, sum_of_squares, and more in one aggregation.

3. Step 3: Compare other options

   stats lacks variance and std deviation; terms groups data; percentiles gives percentile info, not variance.

4. Final Answer:

   Use extended_stats aggregation on price -> Option C

5. Quick Check:

   Extended stats = avg + variance + std deviation [OK]

Hint: Extended stats give avg and spread metrics together [OK]

Common Mistakes:

• Using stats and missing variance
• Confusing terms with stats
• Choosing percentiles for variance