Bucket pattern for time-series data in MongoDB - Step-by-Step Execution

Concept Flow - Bucket pattern for time-series data

Receive time-series data

↓

Group data by time intervals

↓

Create bucket document for each interval

↓

Store multiple data points inside bucket

↓

Query buckets for time range

↓

Extract individual data points from buckets

Data points are grouped into time buckets to store many points in one document, improving write and query efficiency.

Execution Sample

MongoDB

db.sensorData.insertOne({
  bucketStart: ISODate("2024-06-01T00:00:00Z"),
  readings: [
    {time: ISODate("2024-06-01T00:01:00Z"), value: 20},
    {time: ISODate("2024-06-01T00:02:00Z"), value: 21}
  ]
})

Insert a bucket document containing multiple sensor readings within a 1-hour interval.

Execution Table

Step	Action	Data State	Result
1	Receive new data point at 00:01	No buckets yet	Prepare to create bucket for 00:00-01:00
2	Check if bucket for 00:00 exists	No bucket found	Create new bucket document with bucketStart 00:00
3	Insert data point into readings array	Bucket readings empty	Readings array now has 1 data point
4	Receive new data point at 00:02	Bucket for 00:00 exists with 1 reading	Add new data point to readings array
5	Query data for 00:00-01:00	One bucket with 2 readings	Return bucket document with both readings
6	Extract individual readings from bucket	Bucket readings array	Two separate data points available for analysis
7	Receive data point at 01:05	Bucket for 00:00 full, no bucket for 01:00	Create new bucket for 01:00 interval
8	Insert data point into new bucket	New bucket readings empty	Readings array now has 1 data point
9	Query data for 01:00-02:00	One bucket with 1 reading	Return bucket document with that reading
10	End of demonstration	Buckets created and queried	Efficient storage and retrieval achieved

💡 No more data points to process; buckets created and queried successfully.

Variable Tracker

Variable	Start	After Step 3	After Step 4	After Step 7	After Step 8	Final
buckets	{}	{"00:00": [{time: "00:01", value: 20}]}	{"00:00": [{time: "00:01", value: 20}, {time: "00:02", value: 21}]}	{"00:00": [...], "01:00": []}	{"00:00": [...], "01:00": [{time: "01:05", value: "X"}]}	{"00:00": [...], "01:00": [...] }

Key Moments - 3 Insights

Why do we group multiple time-series points into one bucket document?

Grouping reduces the number of documents MongoDB must handle, improving write speed and query efficiency, as shown in steps 2-4 where multiple points go into one bucket.

How do we know which bucket a new data point belongs to?

We calculate the bucket interval from the data point's timestamp, like 00:00-01:00 for points at 00:01 and 00:02, as seen in step 1 and 2.

What happens when a data point falls outside existing buckets?

A new bucket document is created for that time interval, shown in step 7 when a point at 01:05 triggers a new bucket creation.

Visual Quiz - 3 Questions

Look at the execution table, at which step is the first bucket document created?

A Step 3

B Step 1

C Step 2

D Step 4

💡 Hint

Check the 'Action' column for bucket creation in the execution_table.

According to variable_tracker, how many readings are in the bucket after Step 4?

A 1 reading

B 2 readings

C 0 readings

D 3 readings

💡 Hint

Look at the 'buckets' variable state after Step 4 in variable_tracker.

If a new data point at 00:30 is received, which bucket will it be added to?

A Bucket starting at 00:00

B Bucket starting at 01:00

C A new bucket starting at 00:30

D No bucket, data discarded

💡 Hint

Refer to the concept_flow and execution_table steps about bucket intervals.

Concept Snapshot

Bucket pattern groups many time-series points into one document per time interval.
Each bucket document has a start time and an array of readings.
New data points go into the bucket matching their time interval.
Queries retrieve buckets for a time range, then extract points.
This reduces document count and improves performance.

Full Transcript

The bucket pattern for time-series data stores many data points in one document per time interval. When new data arrives, the system checks if a bucket for that time interval exists. If not, it creates one. Then it adds the data point to the bucket's readings array. Queries fetch buckets for a time range and extract individual points. This method improves write and query efficiency by reducing the number of documents handled. The execution table shows steps from receiving data points, creating buckets, inserting readings, to querying buckets. The variable tracker shows how the buckets variable changes as points are added. Key moments clarify why grouping helps, how buckets are chosen, and what happens when data falls outside existing buckets. The visual quiz tests understanding of bucket creation, readings count, and bucket assignment.