Stats and extended stats in Elasticsearch - Time & Space Complexity

Time Complexity: Stats and extended stats

O(n)

Understanding Time Complexity

When using stats and extended stats in Elasticsearch, it's important to know how the time to get results grows as data grows.

We want to understand how the number of documents affects the time Elasticsearch takes to calculate these statistics.

Scenario Under Consideration

Analyze the time complexity of the following Elasticsearch aggregation query.

{
  "size": 0,
  "aggs": {
    "stats_example": {
      "stats": { "field": "price" }
    },
    "extended_stats_example": {
      "extended_stats": { "field": "price" }
    }
  }
}
    

This query calculates basic stats and extended stats on the "price" field for all matching documents.

Identify Repeating Operations

Look at what repeats as Elasticsearch processes the data.

• Primary operation: Scanning each document's "price" field to update stats.
• How many times: Once for every document that matches the query.

How Execution Grows With Input

As the number of documents increases, the work to calculate stats grows in a straight line.

Input Size (n)	Approx. Operations
10	10 operations scanning "price" values
100	100 operations scanning "price" values
1000	1000 operations scanning "price" values

Pattern observation: The number of operations grows directly with the number of documents.

Final Time Complexity

Time Complexity: O(n)

This means the time to compute stats grows in direct proportion to the number of documents.

Common Mistake

[X] Wrong: "Stats calculations are instant no matter how many documents there are."

[OK] Correct: Elasticsearch must look at each document's field value to calculate stats, so more documents mean more work and more time.

Interview Connect

Understanding how stats aggregations scale helps you explain performance in real projects and shows you can think about data size impact clearly.

Self-Check

"What if we added a filter to reduce the documents before stats calculation? How would the time complexity change?"

Practice

1. What does the stats aggregation in Elasticsearch provide by default?

easy

A. Count, min, max, avg, and sum

B. Variance and standard deviation

C. Sum of squares only

D. Median and mode

Solution

1. Step 1: Understand the purpose of stats aggregation

   The stats aggregation provides basic statistical data about numeric fields.

2. Step 2: Identify the default metrics included

   By default, it returns count, min, max, avg, and sum values.

3. Final Answer:

   Count, min, max, avg, and sum -> Option A

4. Quick Check:

   Stats aggregation = basic stats [OK]

Hint: Stats aggregation gives basic stats like count and sum [OK]

Common Mistakes:

• Confusing stats with extended_stats
• Thinking stats includes variance
• Assuming median is included

2. Which of the following is the correct syntax to add an extended stats aggregation named my_stats on the field price?

easy

A. { "aggs": { "my_stats": { "extended_stats": { "field": "price" } } } }

B. { "aggs": { "my_stats": { "stats": { "field": "price" } } } }

C. { "aggs": { "my_stats": { "extended_stats": { "field_name": "price" } } } }

D. { "aggregations": { "my_stats": { "stats": { "field": "price" } } } }

Solution

1. Step 1: Check the aggregation type and field key

   The aggregation type must be extended_stats and the field key must be field.

2. Step 2: Verify the JSON structure

   The correct syntax uses aggs as the root key for aggregations, not aggregations.

3. Final Answer:

   { "aggs": { "my_stats": { "extended_stats": { "field": "price" } } } } -> Option A

4. Quick Check:

   Use extended_stats with field inside aggs [OK]

Hint: Use "aggs" and "extended_stats" with "field" key [OK]

Common Mistakes:

• Using "field_name" instead of "field"
• Using "aggregations" instead of "aggs"
• Confusing stats with extended_stats

3. Given this Elasticsearch aggregation query:

{
  "aggs": {
    "price_stats": {
      "extended_stats": {
        "field": "price"
      }
    }
  }
}

Which of the following metrics will NOT be included in the response?

medium

A. Sum of squares

B. Median

C. Standard deviation

D. Variance

Solution

1. Step 1: Identify metrics returned by extended_stats

   The extended_stats aggregation returns count, min, max, avg, sum, sum_of_squares, variance, and std_deviation.

2. Step 2: Check which metric is missing

   Median is not included in either stats or extended_stats aggregations.

3. Final Answer:

   Median -> Option B

4. Quick Check:

   Median not in extended_stats [OK]

Hint: Extended stats exclude median, include variance and std deviation [OK]

Common Mistakes:

• Assuming median is part of extended_stats
• Confusing sum_of_squares with sum
• Thinking variance is not included

4. You wrote this aggregation:

{
  "aggs": {
    "my_stats": {
      "extended_stats": {
        "field_name": "age"
      }
    }
  }
}

Why does this query fail?

medium

A. The aggregation name cannot be "my_stats"

B. The root key should be "aggregations" instead of "aggs"

C. extended_stats does not support the field "age"

D. The field key should be "field", not "field_name"

Solution

1. Step 1: Check the field key in extended_stats

   The correct key to specify the field is field, not field_name.

2. Step 2: Verify other parts of the query

   The aggregation name "my_stats" is valid, and "aggs" is the correct root key. The field "age" can be used if it exists.

3. Final Answer:

   The field key should be "field", not "field_name" -> Option D

4. Quick Check:

   Use "field" key for field name [OK]

Hint: Use "field" key, not "field_name" in aggregations [OK]

Common Mistakes:

• Using "field_name" instead of "field"
• Changing aggregation root key incorrectly
• Assuming field name is invalid without checking

5. You want to find the average price and also understand the price spread using variance and standard deviation in one query. Which aggregation setup is best?

hard

A. Use stats aggregation on price

B. Use percentiles aggregation on price

C. Use extended_stats aggregation on price

D. Use terms aggregation on price

Solution

1. Step 1: Identify metrics needed

   You want average, variance, and standard deviation, which describe spread and central tendency.

2. Step 2: Choose aggregation that provides all metrics

   extended_stats provides avg, variance, std_deviation, sum_of_squares, and more in one aggregation.

3. Step 3: Compare other options

   stats lacks variance and std deviation; terms groups data; percentiles gives percentile info, not variance.

4. Final Answer:

   Use extended_stats aggregation on price -> Option C

5. Quick Check:

   Extended stats = avg + variance + std deviation [OK]

Hint: Extended stats give avg and spread metrics together [OK]

Common Mistakes:

• Using stats and missing variance
• Confusing terms with stats
• Choosing percentiles for variance