Date histogram in Elasticsearch - Mini Project: Build & Apply

Date histogram

📖 Scenario: You work for a company that collects sales data with timestamps. You want to see how many sales happened each day.

🎯 Goal: Create an Elasticsearch query that groups sales by day using a date histogram aggregation.

📋 What You'll Learn

Create an Elasticsearch query with a date histogram aggregation named sales_per_day.

Use the field sale_date for the date histogram.

Set the interval to day.

Include a match_all query to get all documents.

💡 Why This Matters

🌍 Real World

Date histograms help businesses analyze trends over time, like daily sales or website visits.

💼 Career

Knowing how to write date histogram queries is useful for data analysts and backend developers working with time-series data in Elasticsearch.

1

Create the base query

Create a variable called query and set it to a JSON object with a query key containing a match_all query.

Elasticsearch

query = {
    "query": {
        "match_all": {}
    }
}
# Your code here

Hint

The match_all query returns all documents.

2

Add the date histogram aggregation

Add an aggs key to the query variable. Inside aggs, create a date histogram aggregation named sales_per_day that uses the field sale_date and sets the interval to day.

Elasticsearch

query = {
    "query": {
        "match_all": {}
    },
    # Add aggs here
}
# Your code here

Hint

The date_histogram aggregation groups documents by date intervals.

3

Complete the query structure

Ensure the query variable is a complete JSON object with query and aggs keys as described. No changes needed if already done.

Elasticsearch

query = {
    "query": {
        "match_all": {}
    },
    "aggs": {
        "sales_per_day": {
            "date_histogram": {
                "field": "sale_date",
                "calendar_interval": "day"
            }
        }
    }
}
# Your code here

Hint

Check that the query has both query and aggs keys.

4

Print the final query

Write print(query) to display the complete Elasticsearch query.

Elasticsearch

query = {
    "query": {
        "match_all": {}
    },
    "aggs": {
        "sales_per_day": {
            "date_histogram": {
                "field": "sale_date",
                "calendar_interval": "day"
            }
        }
    }
}

# Print the query
# Your code here

Hint

Use print(query) to see the full query JSON.

Practice

1. What is the primary purpose of a date_histogram aggregation in Elasticsearch?

easy

A. To group documents into buckets based on time intervals

B. To sort documents by date in ascending order

C. To filter documents by a specific date range

D. To count the total number of documents in the index

Solution

1. Step 1: Understand the function of date_histogram

   The date_histogram aggregation groups documents into buckets based on specified time intervals like days or months.

2. Step 2: Compare with other options

   Sorting, filtering, and counting are different operations; only grouping by time intervals matches date_histogram.

3. Final Answer:

   To group documents into buckets based on time intervals -> Option A

4. Quick Check:

   date_histogram = group by time [OK]

Hint: Remember: date_histogram groups data by time intervals [OK]

Common Mistakes:

• Confusing date_histogram with sorting
• Thinking it filters documents
• Assuming it counts all documents without grouping

2. Which of the following is the correct syntax to create a date_histogram aggregation on the field timestamp with a daily interval?

easy

A. "date_histogram": { "field": "timestamp", "interval": "day" }

B. "date_histogram": { "field": "timestamp", "interval": "daily" }

C. "date_histogram": { "field": "timestamp", "interval": "1day" }

D. "date_histogram": { "field": "timestamp", "interval": "days" }

Solution

1. Step 1: Check valid interval values

   Elasticsearch expects intervals like "day", "week", "month" as strings without plural or numeric prefixes.

2. Step 2: Validate each option

   "date_histogram": { "field": "timestamp", "interval": "day" } uses "day" which is correct. Options B, C, and D use invalid interval formats.

3. Final Answer:

   "date_histogram": { "field": "timestamp", "interval": "day" } -> Option A

4. Quick Check:

   Use singular interval names like "day" [OK]

Hint: Use singular interval names like "day", not "daily" or "days" [OK]

Common Mistakes:

• Using plural interval names like "days"
• Using malformed numeric prefixes like "1day"
• Using incorrect interval strings like "daily"

3. Given this Elasticsearch aggregation query snippet:

{
  "aggs": {
    "sales_over_time": {
      "date_histogram": {
        "field": "sale_date",
        "calendar_interval": "month"
      }
    }
  }
}

What will be the result of this aggregation?

medium

A. Buckets grouping sales by each day based on sale_date

B. Buckets grouping sales by each month based on sale_date

C. A syntax error due to missing interval

D. Buckets grouping sales by year ignoring months

Solution

1. Step 1: Identify the interval type

   The query uses calendar_interval": "month", which groups data by calendar months.

2. Step 2: Understand the effect on buckets

   Each bucket will contain documents with sale_date in the same month.

3. Final Answer:

   Buckets grouping sales by each month based on sale_date -> Option B

4. Quick Check:

   calendar_interval month = monthly buckets [OK]

Hint: calendar_interval "month" means monthly buckets [OK]

Common Mistakes:

• Confusing calendar_interval with fixed_interval
• Assuming daily buckets instead of monthly
• Thinking missing interval causes syntax error

4. You wrote this date_histogram aggregation:

{
  "aggs": {
    "by_week": {
      "date_histogram": {
        "field": "created_at",
        "interval": "week"
      }
    }
  }
}

But Elasticsearch returns an error. What is the likely cause?

medium

A. Missing "time_zone" parameter causes error

B. Field "created_at" is missing from the index

C. Using deprecated "interval" instead of "calendar_interval" or "fixed_interval"

D. The interval value "week" is invalid

Solution

1. Step 1: Check interval parameter usage

   Since Elasticsearch 7.2, "interval" is deprecated; use "calendar_interval" or "fixed_interval" instead.

2. Step 2: Identify error cause

   Using "interval" causes an error; other options are less likely or cause different errors.

3. Final Answer:

   Using deprecated "interval" instead of "calendar_interval" or "fixed_interval" -> Option C

4. Quick Check:

   Use calendar_interval or fixed_interval, not interval [OK]

Hint: Replace deprecated "interval" with "calendar_interval" or "fixed_interval" [OK]

Common Mistakes:

• Ignoring deprecation and using "interval"
• Assuming missing field causes this specific error
• Thinking "week" is invalid interval

5. You want to create a date_histogram aggregation that groups documents by week but aligns weeks to start on Monday instead of Sunday. Which parameter should you add to your aggregation?

hard

A. "calendar_interval": "week"

B. "time_zone": "UTC"

C. "week_start": "monday"

D. "offset": "1d"

Solution

1. Step 1: Understand week alignment in date_histogram

   Elasticsearch weeks start on Sunday by default; to shift start day, use the "offset" parameter.

2. Step 2: Identify correct parameter

   Setting "offset": "1d" shifts buckets by one day, making weeks start on Monday.

3. Step 3: Check other options

   "time_zone" adjusts time zones, "week_start" is not a valid parameter, and "calendar_interval" defines interval but not start day.

4. Final Answer:

   "offset": "1d" -> Option D

5. Quick Check:

   Use offset to shift week start day [OK]

Hint: Use offset "1d" to start weeks on Monday [OK]

Common Mistakes:

• Trying to use non-existent "week_start" parameter
• Assuming time_zone changes week start day
• Not using offset to shift buckets