Pipeline mental model (stages flow) in MongoDB - Mini Project: Build & Apply

Build a MongoDB Aggregation Pipeline

📖 Scenario: You work at a small online bookstore. You have a collection of books with details like title, author, genre, and sales. You want to analyze the data to find the total sales per genre.

🎯 Goal: Create a MongoDB aggregation pipeline that groups books by genre and calculates the total sales for each genre.

📋 What You'll Learn

Create a books collection with specific book documents

Add a variable for the minimum sales threshold

Build an aggregation pipeline that filters books with sales above the threshold

Group the filtered books by genre and sum their sales

💡 Why This Matters

🌍 Real World

Aggregation pipelines help analyze and summarize large collections of data in MongoDB, such as sales reports or user activity.

💼 Career

Understanding aggregation pipelines is essential for roles like data analyst, backend developer, or database administrator working with MongoDB.

1

DATA SETUP: Create the books collection with documents

Create a variable called books that is a list of documents with these exact entries: { title: 'The Alchemist', author: 'Paulo Coelho', genre: 'Fiction', sales: 500 }, { title: 'Clean Code', author: 'Robert C. Martin', genre: 'Programming', sales: 300 }, { title: 'The Pragmatic Programmer', author: 'Andrew Hunt', genre: 'Programming', sales: 250 }, { title: '1984', author: 'George Orwell', genre: 'Fiction', sales: 400 }, and { title: 'Deep Work', author: 'Cal Newport', genre: 'Self-help', sales: 150 }.

MongoDB

# Create the books list with the specified documents
# Your code here

Hint

Use a Python list of dictionaries to represent the books collection documents.

2

CONFIGURATION: Set a sales threshold variable

Create a variable called min_sales and set it to 200 to filter books with sales greater than or equal to this value.

MongoDB

books = [
    { 'title': 'The Alchemist', 'author': 'Paulo Coelho', 'genre': 'Fiction', 'sales': 500 },
    { 'title': 'Clean Code', 'author': 'Robert C. Martin', 'genre': 'Programming', 'sales': 300 },
    { 'title': 'The Pragmatic Programmer', 'author': 'Andrew Hunt', 'genre': 'Programming', 'sales': 250 },
    { 'title': '1984', 'author': 'George Orwell', 'genre': 'Fiction', 'sales': 400 },
    { 'title': 'Deep Work', 'author': 'Cal Newport', 'genre': 'Self-help', 'sales': 150 }
]
# Create min_sales variable and set it to 200
# Your code here

Hint

Just create a variable min_sales and assign the number 200.

3

CORE LOGIC: Build the aggregation pipeline stages

Create a variable called pipeline that is a list of two stages: first, a $match stage that filters books with sales greater than or equal to min_sales, and second, a $group stage that groups by genre and calculates the total sales as totalSales.

MongoDB

books = [
    { 'title': 'The Alchemist', 'author': 'Paulo Coelho', 'genre': 'Fiction', 'sales': 500 },
    { 'title': 'Clean Code', 'author': 'Robert C. Martin', 'genre': 'Programming', 'sales': 300 },
    { 'title': 'The Pragmatic Programmer', 'author': 'Andrew Hunt', 'genre': 'Programming', 'sales': 250 },
    { 'title': '1984', 'author': 'George Orwell', 'genre': 'Fiction', 'sales': 400 },
    { 'title': 'Deep Work', 'author': 'Cal Newport', 'genre': 'Self-help', 'sales': 150 }
]
min_sales = 200
# Create the pipeline list with $match and $group stages
# Your code here

Hint

Use a list with two dictionaries: one for $match filtering sales, one for $group grouping by genre and summing sales.

4

COMPLETION: Prepare the final aggregation command

Create a variable called aggregation_command that is a dictionary with the key aggregate set to 'books' and the key pipeline set to the pipeline variable.

MongoDB

books = [
    { 'title': 'The Alchemist', 'author': 'Paulo Coelho', 'genre': 'Fiction', 'sales': 500 },
    { 'title': 'Clean Code', 'author': 'Robert C. Martin', 'genre': 'Programming', 'sales': 300 },
    { 'title': 'The Pragmatic Programmer', 'author': 'Andrew Hunt', 'genre': 'Programming', 'sales': 250 },
    { 'title': '1984', 'author': 'George Orwell', 'genre': 'Fiction', 'sales': 400 },
    { 'title': 'Deep Work', 'author': 'Cal Newport', 'genre': 'Self-help', 'sales': 150 }
]
min_sales = 200
pipeline = [
    { '$match': { 'sales': { '$gte': min_sales } } },
    { '$group': { '_id': '$genre', 'totalSales': { '$sum': '$sales' } } }
]
# Create aggregation_command dictionary with keys 'aggregate' and 'pipeline'
# Your code here

Hint

Create a dictionary with keys aggregate and pipeline using the variable pipeline.

Practice

1. What does each stage in a MongoDB aggregation pipeline do?

easy

A. Runs a SQL query on the data

B. Creates a new database

C. Transforms or filters data before passing it to the next stage

D. Deletes the entire collection

Solution

1. Step 1: Understand the pipeline concept

   Each stage in a pipeline processes data step-by-step, modifying or filtering it.

2. Step 2: Identify the role of stages

   Stages do not create databases or delete collections; they only transform or filter data.

3. Final Answer:

   Transforms or filters data before passing it to the next stage -> Option C

4. Quick Check:

   Pipeline stages transform or filter data [OK]

Hint: Remember: pipeline stages change data step-by-step [OK]

Common Mistakes:

• Thinking stages create or delete collections
• Confusing pipeline with database commands
• Assuming stages run SQL queries

2. Which of the following is the correct syntax to start an aggregation pipeline in MongoDB?

easy

A. db.collection.find({ $match: { status: 'A' } })

B. db.collection.update({ $match: { status: 'A' } })

C. db.collection.insert([{ $match: { status: 'A' } }])

D. db.collection.aggregate([ { $match: { status: 'A' } } ])

Solution

1. Step 1: Identify aggregation pipeline syntax

   The aggregation pipeline uses the aggregate() method with an array of stages.

2. Step 2: Compare options

   Only db.collection.aggregate([ { $match: { status: 'A' } } ]) uses aggregate() with a pipeline stage ($match). Others use find, insert, or update incorrectly.

3. Final Answer:

   db.collection.aggregate([ { $match: { status: 'A' } } ]) -> Option D

4. Quick Check:

   aggregate() starts pipelines [OK]

Hint: aggregate() method always starts a pipeline [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipelines
• Trying to insert pipeline stages
• Using update() with pipeline syntax

3. Given the pipeline: [{ $match: { age: { $gte: 18 } } }, { $group: { _id: '$city', count: { $sum: 1 } } }], what does the output represent?

medium

A. Count of people grouped by city who are 18 or older

B. List of all people younger than 18

C. Sum of ages for each city

D. All documents without filtering

Solution

1. Step 1: Analyze the $match stage

   This stage filters documents where age is 18 or more.

2. Step 2: Analyze the $group stage

   Groups filtered documents by city and counts how many per city.

3. Final Answer:

   Count of people grouped by city who are 18 or older -> Option A

4. Quick Check:

   $match filters, $group counts by city [OK]

Hint: $match filters, $group aggregates data [OK]

Common Mistakes:

• Confusing $sum with sum of ages
• Ignoring the filter condition
• Assuming output lists all people

4. What is wrong with this pipeline? [{ $match: { score: { $gt: 50 } } }, { $project: { score: 1, name: 1 } }, { $match: { name: 'Alice' } }]

medium

A. Using two $match stages is not allowed

B. The pipeline syntax is correct and will work as expected

C. The $project stage must include _id field explicitly

D. The second $match stage should come before $project

Solution

1. Step 1: Check multiple $match stages

   MongoDB allows multiple $match stages anywhere in the pipeline.

2. Step 2: Check $project usage

   $project can select fields; _id is included by default unless excluded explicitly.

3. Final Answer:

   The pipeline syntax is correct and will work as expected -> Option B

4. Quick Check:

   Multiple $match and $project usage is valid [OK]

Hint: Multiple $match stages are allowed anywhere [OK]

Common Mistakes:

• Thinking multiple $match stages are invalid
• Assuming _id must be included explicitly
• Believing $project must come last

5. You want to find the average price of products in each category but only for products with stock greater than 0. Which pipeline correctly achieves this?

hard

A. [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

B. [{ $group: { _id: '$category', avgPrice: { $avg: '$price' } } }, { $match: { stock: { $gt: 0 } } }]

C. [{ $project: { category: 1, price: 1, stock: 1 } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

D. [{ $match: { price: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$stock' } } }]

Solution

1. Step 1: Filter products with stock > 0 first

   The $match stage must come first to filter only products in stock.

2. Step 2: Group by category and calculate average price

   Use $group with _id as category and $avg on price to get average price per category.

3. Final Answer:

   [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }] -> Option A

4. Quick Check:

   Filter then group for correct average [OK]

Hint: Filter first, then group to aggregate correctly [OK]

Common Mistakes:

• Grouping before filtering causes wrong results
• Averaging wrong fields
• Filtering on price instead of stock