Bucket pattern for time-series data in MongoDB

Introduction

The bucket pattern groups many time-series records into one document to save space and improve query speed.

When you collect sensor readings every second and want to store them efficiently.

When you track user activity logs over time and want to reduce the number of documents.

When you monitor server metrics and want faster queries on recent data.

When you want to reduce storage costs by grouping many small events.

When you need to quickly find data for a specific time range.

Syntax

MongoDB

db.collection.aggregate([
  {
    $bucket: {
      groupBy: "$timeField",
      boundaries: [start1, start2, ..., startN],
      default: "other",
      output: {
        count: { $sum: 1 },
        data: { $push: "$dataField" }
      }
    }
  }
])

$bucket groups documents by ranges you define in boundaries.

The groupBy field is usually a date or time field.

Examples

This groups readings into January and February buckets, counting and collecting values.

MongoDB

db.readings.aggregate([
  {
    $bucket: {
      groupBy: "$timestamp",
      boundaries: [ISODate("2024-01-01"), ISODate("2024-02-01"), ISODate("2024-03-01")],
      default: "Other",
      output: {
        count: { $sum: 1 },
        readings: { $push: "$value" }
      }
    }
  }
])

This groups events by hour intervals (0-1h, 1-2h, 2-3h) after midnight.

MongoDB

db.logs.aggregate([
  {
    $bucket: {
      groupBy: "$eventTime",
      boundaries: [0, 3600, 7200, 10800],
      default: "Other",
      output: {
        events: { $push: "$event" },
        total: { $sum: 1 }
      }
    }
  }
])

Sample Program

This example inserts temperature readings at different times and groups them into 1-hour buckets, counting readings and collecting their values.

MongoDB

db.temperatureReadings.insertMany([
  { timestamp: ISODate("2024-06-01T00:10:00Z"), value: 22 },
  { timestamp: ISODate("2024-06-01T00:20:00Z"), value: 23 },
  { timestamp: ISODate("2024-06-01T01:05:00Z"), value: 21 },
  { timestamp: ISODate("2024-06-01T01:45:00Z"), value: 20 },
  { timestamp: ISODate("2024-06-01T02:15:00Z"), value: 19 }
])

db.temperatureReadings.aggregate([
  {
    $bucket: {
      groupBy: "$timestamp",
      boundaries: [
        ISODate("2024-06-01T00:00:00Z"),
        ISODate("2024-06-01T01:00:00Z"),
        ISODate("2024-06-01T02:00:00Z"),
        ISODate("2024-06-01T03:00:00Z")
      ],
      default: "Other",
      output: {
        count: { $sum: 1 },
        values: { $push: "$value" }
      }
    }
  }
])

Important Notes

The boundaries array must be sorted and cover all expected time ranges.

Documents with groupBy values outside boundaries go to the default bucket.

This pattern reduces the number of documents and speeds up queries on time ranges.

Summary

The bucket pattern groups many time-series records into fewer documents.

It uses $bucket with time boundaries to organize data.

This helps save space and makes queries on time ranges faster.

Practice

1.

What is the main purpose of using the bucket pattern in MongoDB for time-series data?

easy

A. To group many time-series records into fewer documents for efficiency

B. To delete old time-series data automatically

C. To create indexes on every single time record

D. To convert time-series data into text format

Solution

1. Step 1: Understand the bucket pattern concept

   The bucket pattern groups multiple time-series records into fewer documents to reduce storage and improve query speed.

2. Step 2: Compare options with the bucket pattern purpose

   Only To group many time-series records into fewer documents for efficiency correctly describes grouping records into fewer documents for efficiency.

3. Final Answer:

   To group many time-series records into fewer documents for efficiency -> Option A

4. Quick Check:

   Bucket pattern groups records = A [OK]

Hint: Bucket pattern groups many records into fewer documents [OK]

Common Mistakes:

• Thinking bucket pattern deletes data automatically
• Confusing bucket pattern with indexing every record
• Believing bucket pattern changes data format

2.

Which of the following is the correct way to use $bucket to group documents by time ranges in MongoDB?

{ $bucket: { groupBy: "$timestamp", boundaries: [ISODate("2024-01-01"), ISODate("2024-02-01"), ISODate("2024-03-01")], default: "Other" } }

easy

A. Using $bucket without groupBy field

B. Using $bucket without boundaries array

C. Using $bucket with string boundaries instead of dates

D. Using $bucket with groupBy on timestamp and specifying boundaries as dates

Solution

1. Step 1: Check the required fields for $bucket

   The $bucket stage requires a groupBy field and a boundaries array to define bucket ranges.

2. Step 2: Validate the boundaries and groupBy usage

   Using $bucket with groupBy on timestamp and specifying boundaries as dates correctly uses groupBy on timestamp and boundaries as ISODate objects, which is valid for time-series bucketing.

3. Final Answer:

   Using $bucket with groupBy on timestamp and specifying boundaries as dates -> Option D

4. Quick Check:

   $bucket needs groupBy and boundaries = A [OK]

Hint: Always include groupBy and boundaries in $bucket stage [OK]

Common Mistakes:

• Omitting boundaries array causes error
• Using strings instead of date objects for time boundaries
• Missing groupBy field in $bucket stage

3.

Given the following aggregation pipeline using $bucket on a collection with timestamps:

[{ $bucket: { groupBy: "$timestamp", boundaries: [ISODate("2024-01-01"), ISODate("2024-02-01"), ISODate("2024-03-01")], default: "Other", output: { count: { $sum: 1 } } } }]

What will be the output if the collection has 5 documents with timestamps in January 2024 and 3 documents in March 2024?

medium

A. [{_id: ISODate("2024-01-01"), count: 5}, {_id: ISODate("2024-02-01"), count: 0}, {_id: "Other", count: 3}]

B. [{_id: ISODate("2024-01-01"), count: 5}, {_id: ISODate("2024-02-01"), count: 3}]

C. [{_id: ISODate("2024-01-01"), count: 8}]

D. [{_id: "Other", count: 8}]

Solution

1. Step 1: Understand bucket boundaries and grouping

   Documents with timestamps from 2024-01-01 to before 2024-02-01 go to first bucket; 2024-02-01 to before 2024-03-01 to second; others to default "Other".

2. Step 2: Assign documents to buckets based on timestamps

   5 docs in January fall in first bucket; 3 docs in March fall outside boundaries, so go to "Other"; second bucket has 0 docs.

3. Final Answer:

   [{_id: ISODate("2024-01-01"), count: 5}, {_id: ISODate("2024-02-01"), count: 0}, {_id: "Other", count: 3}] -> Option A

4. Quick Check:

   Docs in Jan = 5, in Other = 3, Feb bucket empty = C [OK]

Hint: Check which bucket each timestamp fits, others go to default [OK]

Common Mistakes:

• Ignoring the default bucket for out-of-bound timestamps
• Assuming March docs fall in February bucket
• Summing all docs into one bucket

4.

Consider this aggregation pipeline snippet using $bucket:

{ $bucket: { groupBy: "$time", boundaries: [ISODate("2024-01-01")], output: { count: { $sum: 1 } } } }

What is the error in this pipeline?

medium

A. The groupBy field name is incorrect; it should be 'timestamp'

B. The boundaries array is missing the upper boundary for the last bucket

C. The boundaries array must have at least two elements defining ranges

D. The output field 'count' is not allowed in $bucket stage

Solution

1. Step 1: Check boundaries array requirements

   The boundaries array must have at least two elements to define bucket ranges properly, including an upper boundary for the last bucket.

2. Step 2: Identify missing upper boundary

   The boundaries array has only one date, so it cannot define any bucket ranges, causing an error.

3. Final Answer:

   The boundaries array must have at least two elements defining ranges -> Option C

4. Quick Check:

   Boundaries must have at least 2 elements = A [OK]

Hint: Boundaries array must have one more element than buckets [OK]

Common Mistakes:

• Using boundaries array with fewer than two elements
• Assuming groupBy field name must be 'timestamp'
• Thinking output fields are not allowed in $bucket

5.

You want to store sensor readings every minute for a month using the bucket pattern. Each bucket should hold data for one hour. Which boundaries array correctly defines the buckets for January 1, 2024?

hard

A. [ISODate("2024-01-01T00:00:00Z"), ISODate("2024-01-01T00:59:59Z"), ISODate("2024-01-01T01:59:59Z")]

B. [ISODate("2024-01-01T00:00:00Z"), ISODate("2024-01-01T01:00:00Z"), ISODate("2024-01-01T02:00:00Z"), ..., ISODate("2024-01-01T23:00:00Z"), ISODate("2024-01-02T00:00:00Z")]

C. [ISODate("2024-01-01"), ISODate("2024-01-02")]

D. [ISODate("2024-01-01T00:00:00Z"), ISODate("2024-01-01T00:30:00Z"), ISODate("2024-01-01T01:00:00Z")]

Solution

1. Step 1: Understand bucket boundaries for hourly buckets

   Each bucket covers one hour, so boundaries must be every hour mark from midnight to next day midnight.

2. Step 2: Check options for correct hourly boundaries

   [ISODate("2024-01-01T00:00:00Z"), ISODate("2024-01-01T01:00:00Z"), ISODate("2024-01-01T02:00:00Z"), ..., ISODate("2024-01-01T23:00:00Z"), ISODate("2024-01-02T00:00:00Z")] lists hourly ISODate boundaries from 00:00 to 24:00, correctly defining 24 one-hour buckets.

3. Final Answer:

   [ISODate("2024-01-01T00:00:00Z"), ISODate("2024-01-01T01:00:00Z"), ISODate("2024-01-01T02:00:00Z"), ..., ISODate("2024-01-01T23:00:00Z"), ISODate("2024-01-02T00:00:00Z")] -> Option B

4. Quick Check:

   Hourly boundaries every hour for 24 hours = D [OK]

Hint: Boundaries must mark start of each hour plus next day start [OK]

Common Mistakes:

• Using half-hour or irregular intervals for hourly buckets
• Using only start and end of day as boundaries
• Using end times like 00:59:59 instead of next hour start