Why the aggregation pipeline is needed in MongoDB - See It in Action

Understanding Why the Aggregation Pipeline is Needed in MongoDB

📖 Scenario: Imagine you run a small online bookstore. You have a collection of books with details like title, author, price, and number of copies sold. You want to find out which authors sold the most books and the total revenue they generated.

🎯 Goal: Build a simple aggregation pipeline in MongoDB to calculate total copies sold and total revenue per author.

📋 What You'll Learn

Create a collection called books with sample book documents

Add a variable to hold the minimum number of copies sold to consider

Write an aggregation pipeline to group books by author and calculate total copies sold and revenue

Filter authors who sold more than the minimum copies threshold

💡 Why This Matters

🌍 Real World

Online stores and businesses often need to analyze sales data to find top sellers and revenue contributors.

💼 Career

Understanding aggregation pipelines is essential for data analysts and backend developers working with MongoDB to generate reports and insights.

1

DATA SETUP: Create the books collection with sample documents

Create a variable called books that holds a list of three book documents. Each document should have these exact fields and values: { title: 'Book A', author: 'Author 1', price: 10, copiesSold: 100 }, { title: 'Book B', author: 'Author 2', price: 15, copiesSold: 50 }, and { title: 'Book C', author: 'Author 1', price: 20, copiesSold: 70 }.

MongoDB

books = [
    # Your code here
]

Hint

Use a list of dictionaries with the exact keys and values given.

2

CONFIGURATION: Set a minimum copies sold threshold

Create a variable called min_copies_sold and set it to 100. This will be the minimum total copies sold for authors to be included in the results.

MongoDB

books = [
    { 'title': 'Book A', 'author': 'Author 1', 'price': 10, 'copiesSold': 100 },
    { 'title': 'Book B', 'author': 'Author 2', 'price': 15, 'copiesSold': 50 },
    { 'title': 'Book C', 'author': 'Author 1', 'price': 20, 'copiesSold': 70 }
]

# Your code here

Hint

Just assign the number 100 to the variable min_copies_sold.

3

CORE LOGIC: Write the aggregation pipeline to group by author and calculate totals

Create a variable called pipeline that holds a list of aggregation stages. The pipeline should: 1) group documents by author, 2) calculate total copies sold as totalCopies, and 3) calculate total revenue as totalRevenue (price multiplied by copies sold summed). Use $group with _id as $author, and use $sum for totals.

MongoDB

books = [
    { 'title': 'Book A', 'author': 'Author 1', 'price': 10, 'copiesSold': 100 },
    { 'title': 'Book B', 'author': 'Author 2', 'price': 15, 'copiesSold': 50 },
    { 'title': 'Book C', 'author': 'Author 1', 'price': 20, 'copiesSold': 70 }
]

min_copies_sold = 100

# Your code here

Hint

Use $group with _id as $author. Use $sum to add copiesSold and price * copiesSold.

4

COMPLETION: Add a match stage to filter authors by minimum copies sold

Add a $match stage to the pipeline list that filters authors with totalCopies greater than or equal to min_copies_sold. Append this stage after the $group stage.

MongoDB

books = [
    { 'title': 'Book A', 'author': 'Author 1', 'price': 10, 'copiesSold': 100 },
    { 'title': 'Book B', 'author': 'Author 2', 'price': 15, 'copiesSold': 50 },
    { 'title': 'Book C', 'author': 'Author 1', 'price': 20, 'copiesSold': 70 }
]

min_copies_sold = 100

pipeline = [
    {
        '$group': {
            '_id': '$author',
            'totalCopies': { '$sum': '$copiesSold' },
            'totalRevenue': { '$sum': { '$multiply': [ '$price', '$copiesSold' ] } }
        }
    },
    # Your code here
]

Hint

Append a $match stage that filters where totalCopies is greater than or equal to min_copies_sold.

Practice

1. Why do we use the aggregation pipeline in MongoDB?

easy

A. To process and transform data step-by-step inside the database

B. To create new databases

C. To backup the database automatically

D. To connect MongoDB with other databases

Solution

1. Step 1: Understand the purpose of aggregation pipeline

   The aggregation pipeline is designed to process data in stages, allowing filtering, grouping, and transforming data inside MongoDB.

2. Step 2: Compare options with the pipeline's purpose

   Creating databases, backups, or connecting databases are not functions of the aggregation pipeline.

3. Final Answer:

   To process and transform data step-by-step inside the database -> Option A

4. Quick Check:

   Aggregation pipeline = step-by-step data processing [OK]

Hint: Aggregation pipeline processes data inside MongoDB stepwise [OK]

Common Mistakes:

• Confusing aggregation pipeline with database creation
• Thinking it handles backups
• Assuming it connects different databases

2. Which of the following is the correct syntax to start an aggregation pipeline on a collection named orders?

easy

A. db.orders.findAggregate([ { $match: { status: 'shipped' } } ])

B. db.orders.find({ $match: { status: 'shipped' } })

C. db.orders.aggregate({ $match: { status: 'shipped' } })

D. db.orders.aggregate([ { $match: { status: 'shipped' } } ])

Solution

1. Step 1: Identify the correct aggregation method

   The aggregation pipeline uses the aggregate() method with an array of stages.

2. Step 2: Check the syntax of each option

   db.orders.aggregate([ { $match: { status: 'shipped' } } ]) correctly uses aggregate with an array containing a $match stage. The other options use invalid methods like findAggregate(), misuse find() with aggregation stages, or pass an object instead of an array to aggregate().

3. Final Answer:

   db.orders.aggregate([ { $match: { status: 'shipped' } } ]) -> Option D

4. Quick Check:

   aggregate() takes an array of stages [OK]

Hint: aggregate() needs an array of stages, not an object [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipelines
• Passing an object instead of an array to aggregate()
• Using non-existent methods like findAggregate()

3. Given the collection sales with documents:

{ "item": "apple", "qty": 5 }

{ "item": "banana", "qty": 10 }

{ "item": "apple", "qty": 7 }

What will this aggregation pipeline return?

db.sales.aggregate([ { $match: { item: "apple" } }, { $group: { _id: "$item", totalQty: { $sum: "$qty" } } } ])

medium

A. [{ "_id": "apple", "totalQty": 7 }]

B. [{ "_id": "banana", "totalQty": 10 }]

C. [{ "_id": "apple", "totalQty": 12 }]

D. []

Solution

1. Step 1: Filter documents with $match

   The pipeline first selects documents where item is "apple". These are the first and third documents with quantities 5 and 7.

2. Step 2: Group and sum quantities with $group

   The pipeline groups by item and sums the qty values: 5 + 7 = 12.

3. Final Answer:

   [{ "_id": "apple", "totalQty": 12 }] -> Option C

4. Quick Check:

   Sum qty for apples = 12 [OK]

Hint: Match filters first, then group sums values [OK]

Common Mistakes:

• Summing all items instead of filtered ones
• Confusing _id field in group stage
• Ignoring the $match filter

4. What is wrong with this aggregation pipeline?

db.products.aggregate({ $sort: { price: 1 } })

medium

A. The field name price is invalid

B. The $sort stage must be inside an array

C. aggregate() cannot sort data

D. Missing a $match stage before $sort

Solution

1. Step 1: Check aggregate() method syntax

   The aggregate() method requires an array of stages, even if there is only one stage.

2. Step 2: Identify the error in the pipeline

   The pipeline is passed as an object instead of an array, causing a syntax error.

3. Final Answer:

   The $sort stage must be inside an array -> Option B

4. Quick Check:

   aggregate() needs array of stages [OK]

Hint: Always wrap pipeline stages in an array [OK]

Common Mistakes:

• Passing pipeline as object instead of array
• Assuming aggregate() can't sort
• Thinking $match is required before $sort

5. You want to find the average quantity sold per item from a transactions collection. Which aggregation pipeline correctly achieves this?

hard

A. db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ])

B. db.transactions.aggregate([ { $match: { qty: { $avg: true } } } ])

C. db.transactions.aggregate([ { $sort: { qty: 1 } }, { $avg: "$qty" } ])

D. db.transactions.aggregate([ { $group: { _id: "$qty", avgQty: { $sum: "$item" } } } ])

Solution

1. Step 1: Understand the goal

   We want to group documents by item and calculate the average of qty for each group.

2. Step 2: Analyze each option

   db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) groups by item and uses $avg on qty, which is correct. db.transactions.aggregate([ { $match: { qty: { $avg: true } } } ]) misuses $avg inside $match. db.transactions.aggregate([ { $sort: { qty: 1 } }, { $avg: "$qty" } ]) incorrectly uses $avg as a pipeline stage. db.transactions.aggregate([ { $group: { _id: "$qty", avgQty: { $sum: "$item" } } } ]) groups by qty and sums item, which is wrong.

3. Final Answer:

   db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) -> Option A

4. Quick Check:

   Group by item and average qty = db.transactions.aggregate([ { $group: { _id: "$item", avgQty: { $avg: "$qty" } } } ]) [OK]

Hint: Use $group with $avg to calculate averages per group [OK]

Common Mistakes:

• Using $avg inside $match stage
• Trying to use $avg as a pipeline stage
• Grouping by wrong field or summing instead of averaging