Bucket pattern for time-series data in MongoDB - Mini Project: Build & Apply

Bucket Pattern for Time-Series Data in MongoDB

📖 Scenario: You are managing a temperature sensor system that records temperature readings every minute. To efficiently store and query this time-series data, you want to group readings into daily buckets in MongoDB.

🎯 Goal: Build a MongoDB collection that uses the bucket pattern to store temperature readings grouped by day. You will create the initial data, configure the bucket size, write the aggregation query to group data by day, and finalize the bucketed collection structure.

📋 What You'll Learn

Create a collection called temperature_readings with sample documents containing timestamp and temperature fields.

Define a variable bucketSize representing the number of milliseconds in one day.

Write an aggregation pipeline that groups readings by day using the bucket pattern.

Add the final stage to output the bucketed data with day and readings fields.

💡 Why This Matters

🌍 Real World

Time-series data like sensor readings, logs, or financial data often need efficient storage and querying. The bucket pattern groups data into time intervals to reduce document count and improve query speed.

💼 Career

Understanding how to organize and query time-series data is valuable for roles in data engineering, backend development, and database administration.

1

Create sample temperature readings

Insert three documents into the temperature_readings collection with these exact fields and values: { timestamp: new Date('2024-06-01T10:00:00Z'), temperature: 22 }, { timestamp: new Date('2024-06-01T15:00:00Z'), temperature: 24 }, and { timestamp: new Date('2024-06-02T09:00:00Z'), temperature: 20 }.

MongoDB

// Insert the three temperature reading documents into temperature_readings
// Your code here

Need a hint?

Use insertMany on db.temperature_readings with an array of objects.

2

Define the bucket size for one day

Create a variable called bucketSize and set it to the number of milliseconds in one day (24 hours).

MongoDB

db.temperature_readings.insertMany([
  { timestamp: new Date('2024-06-01T10:00:00Z'), temperature: 22 },
  { timestamp: new Date('2024-06-01T15:00:00Z'), temperature: 24 },
  { timestamp: new Date('2024-06-02T09:00:00Z'), temperature: 20 }
])

// Define bucketSize as milliseconds in one day
// Your code here

Need a hint?

Calculate milliseconds by multiplying hours, minutes, seconds, and 1000.

3

Write aggregation to group readings by day

Write an aggregation pipeline on temperature_readings that groups documents by day using $subtract and $mod on timestamp with bucketSize. Use $group with _id as the bucket start time and collect readings in an array called readings.

MongoDB

db.temperature_readings.insertMany([
  { timestamp: new Date('2024-06-01T10:00:00Z'), temperature: 22 },
  { timestamp: new Date('2024-06-01T15:00:00Z'), temperature: 24 },
  { timestamp: new Date('2024-06-02T09:00:00Z'), temperature: 20 }
])

const bucketSize = 24 * 60 * 60 * 1000;

// Write aggregation pipeline to group by day using bucket pattern
// Your code here

Need a hint?

Use $group with _id as the bucket start timestamp calculated by subtracting the remainder of timestamp modulo bucketSize.

4

Add final stage to format bucketed data

Add a $project stage to the aggregation pipeline that outputs day as a Date from _id and includes the readings array. Assign the complete pipeline to a variable called bucketedData using db.temperature_readings.aggregate(pipeline).

MongoDB

db.temperature_readings.insertMany([
  { timestamp: new Date('2024-06-01T10:00:00Z'), temperature: 22 },
  { timestamp: new Date('2024-06-01T15:00:00Z'), temperature: 24 },
  { timestamp: new Date('2024-06-02T09:00:00Z'), temperature: 20 }
])

const bucketSize = 24 * 60 * 60 * 1000;

const pipeline = [
  {
    $group: {
      _id: {
        $subtract: [
          { $toLong: "$timestamp" },
          { $mod: [{ $toLong: "$timestamp" }, bucketSize] }
        ]
      },
      readings: { $push: "$temperature" }
    }
  }
];

// Add $project stage to convert _id to day and include readings
// Assign aggregation result to bucketedData
// Your code here

Need a hint?

Use $project to rename _id to day as a date and keep readings. Then assign the aggregation cursor to bucketedData.