Why the aggregation pipeline is needed in MongoDB

Introduction

The aggregation pipeline helps you process and analyze data step-by-step inside the database. It makes complex data tasks easier and faster.

When you want to calculate totals or averages from many records.

When you need to filter and group data to find patterns.

When you want to transform data into a new shape or format.

When you want to combine data from multiple collections.

When you want to sort and limit results based on conditions.

Syntax

MongoDB

db.collection.aggregate([
  { stage1 },
  { stage2 },
  ...
])

Each stage is an object that performs one operation on the data.

Stages run in order, passing results from one to the next.

Examples

This example filters sales with status 'A' and then groups them by item to sum the amounts.

MongoDB

db.sales.aggregate([
  { $match: { status: "A" } },
  { $group: { _id: "$item", total: { $sum: "$amount" } } }
])

This example sorts orders by date descending and returns the latest 5 orders.

MongoDB

db.orders.aggregate([
  { $sort: { date: -1 } },
  { $limit: 5 }
])

Sample Program

This query finds students with grades 70 or higher, groups them by class, calculates the average grade per class, and sorts classes by average score descending.

MongoDB

db.students.aggregate([
  { $match: { grade: { $gte: 70 } } },
  { $group: { _id: "$class", averageScore: { $avg: "$grade" } } },
  { $sort: { averageScore: -1 } }
])

Important Notes

The aggregation pipeline is powerful for data analysis inside MongoDB without moving data outside.

Each stage can do different things like filtering, grouping, sorting, or reshaping data.

Summary

The aggregation pipeline processes data step-by-step inside the database.

It helps with filtering, grouping, sorting, and transforming data easily.

Using it can make data analysis faster and simpler.

Practice

1. Why do we use the aggregation pipeline in MongoDB?

easy

A. To process and transform data step-by-step inside the database

B. To create new databases

C. To backup the database automatically

D. To connect MongoDB with other databases

Solution

1. Step 1: Understand the purpose of aggregation pipeline

   The aggregation pipeline is designed to process data in stages, allowing filtering, grouping, and transforming data inside MongoDB.

2. Step 2: Compare options with the pipeline's purpose

   Creating databases, backups, or connecting databases are not functions of the aggregation pipeline.

3. Final Answer:

   To process and transform data step-by-step inside the database -> Option A

4. Quick Check:

   Aggregation pipeline = step-by-step data processing [OK]

Hint: Aggregation pipeline processes data inside MongoDB stepwise [OK]

Common Mistakes:

• Confusing aggregation pipeline with database creation
• Thinking it handles backups
• Assuming it connects different databases

2. Which of the following is the correct syntax to start an aggregation pipeline on a collection named orders?

easy

A. db.orders.findAggregate([ { $match: { status: 'shipped' } } ])

B. db.orders.find({ $match: { status: 'shipped' } })

C. db.orders.aggregate({ $match: { status: 'shipped' } })

D. db.orders.aggregate([ { $match: { status: 'shipped' } } ])

Solution

1. Step 1: Identify the correct aggregation method

   The aggregation pipeline uses the aggregate() method with an array of stages.

2. Step 2: Check the syntax of each option

   db.orders.aggregate([ { $match: { status: 'shipped' } } ]) correctly uses aggregate with an array containing a $match stage. The other options use invalid methods like findAggregate(), misuse find() with aggregation stages, or pass an object instead of an array to aggregate().

3. Final Answer:

   db.orders.aggregate([ { $match: { status: 'shipped' } } ]) -> Option D

4. Quick Check:

   aggregate() takes an array of stages [OK]

Hint: aggregate() needs an array of stages, not an object [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipelines
• Passing an object instead of an array to aggregate()
• Using non-existent methods like findAggregate()

3. Given the collection sales with documents:

{ "item": "apple", "qty": 5 }

{ "item": "banana", "qty": 10 }

{ "item": "apple", "qty": 7 }

What will this aggregation pipeline return?

db.sales.aggregate([ { $match: { item: "apple" } }, { $group: { _id: "$item", totalQty: { $sum: "$qty" } } } ])

medium

A. [{ "_id": "apple", "totalQty": 7 }]

B. [{ "_id": "banana", "totalQty": 10 }]

C. [{ "_id": "apple", "totalQty": 12 }]

D. []

Solution

1. Step 1: Filter documents with $match

   The pipeline first selects documents where item is "apple". These are the first and third documents with quantities 5 and 7.

2. Step 2: Group and sum quantities with $group

   The pipeline groups by item and sums the qty values: 5 + 7 = 12.

3. Final Answer:

   [{ "_id": "apple", "totalQty": 12 }] -> Option C

4. Quick Check:

   Sum qty for apples = 12 [OK]

Hint: Match filters first, then group sums values [OK]

Common Mistakes:

• Summing all items instead of filtered ones
• Confusing _id field in group stage
• Ignoring the $match filter

4. What is wrong with this aggregation pipeline?

db.products.aggregate({ $sort: { price: 1 } })

medium

A. The field name price is invalid

B. The $sort stage must be inside an array

C. aggregate() cannot sort data

D. Missing a $match stage before $sort

Solution

1. Step 1: Check aggregate() method syntax

   The aggregate() method requires an array of stages, even if there is only one stage.

2. Step 2: Identify the error in the pipeline

   The pipeline is passed as an object instead of an array, causing a syntax error.

3. Final Answer:

   The $sort stage must be inside an array -> Option B

4. Quick Check:

   aggregate() needs array of stages [OK]

Hint: Always wrap pipeline stages in an array [OK]

Common Mistakes:

• Passing pipeline as object instead of array
• Assuming aggregate() can't sort
• Thinking $match is required before $sort

5. You want to find the average quantity sold per item from a transactions collection. Which aggregation pipeline correctly achieves this?

hard

A. db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ])

B. db.transactions.aggregate([ { $match: { qty: { $avg: true } } } ])

C. db.transactions.aggregate([ { $sort: { qty: 1 } }, { $avg: "$qty" } ])

D. db.transactions.aggregate([ { $group: { _id: "$qty", avgQty: { $sum: "$item" } } } ])

Solution

1. Step 1: Understand the goal

   We want to group documents by item and calculate the average of qty for each group.

2. Step 2: Analyze each option

   db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) groups by item and uses $avg on qty, which is correct. db.transactions.aggregate([ { $match: { qty: { $avg: true } } } ]) misuses $avg inside $match. db.transactions.aggregate([ { $sort: { qty: 1 } }, { $avg: "$qty" } ]) incorrectly uses $avg as a pipeline stage. db.transactions.aggregate([ { $group: { _id: "$qty", avgQty: { $sum: "$item" } } } ]) groups by qty and sums item, which is wrong.

3. Final Answer:

   db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) -> Option A

4. Quick Check:

   Group by item and average qty = db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) [OK]

Hint: Use $group with $avg to calculate averages per group [OK]

Common Mistakes:

• Using $avg inside $match stage
• Trying to use $avg as a pipeline stage
• Grouping by wrong field or summing instead of averaging