Pipeline mental model (stages flow) in MongoDB

Introduction

A pipeline helps process data step-by-step, like an assembly line, making it easy to transform and analyze data in order.

When you want to filter data before doing calculations.

When you need to sort data and then group it by categories.

When you want to add new information to each data item based on existing fields.

When you want to combine multiple operations on data in a clear sequence.

When you want to prepare data for reports by shaping it step-by-step.

Syntax

MongoDB

db.collection.aggregate([
  { stage1 },
  { stage2 },
  ...
])

Each stage is an object that performs one operation on the data.

Stages run one after another, passing results from one to the next.

Examples

First filters sales with status 'A', then groups by item and sums amounts.

MongoDB

db.sales.aggregate([
  { $match: { status: "A" } },
  { $group: { _id: "$item", total: { $sum: "$amount" } } }
])

Sorts users by age descending, then takes the top 5.

MongoDB

db.users.aggregate([
  { $sort: { age: -1 } },
  { $limit: 5 }
])

Sample Program

This pipeline finds all shipped orders, groups them by customer summing prices, then sorts customers by total spent descending.

MongoDB

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $group: { _id: "$customer", totalSpent: { $sum: "$price" } } },
  { $sort: { totalSpent: -1 } }
])

Important Notes

Think of each stage as a step in a recipe; the output of one step is the input for the next.

You can add as many stages as needed to get the final result you want.

Order matters: changing the order of stages can change the result or performance.

Summary

A pipeline processes data in stages, one after another.

Each stage transforms or filters the data before passing it on.

Using pipelines helps organize complex data tasks clearly and efficiently.

Practice

1. What does each stage in a MongoDB aggregation pipeline do?

easy

A. Runs a SQL query on the data

B. Creates a new database

C. Transforms or filters data before passing it to the next stage

D. Deletes the entire collection

Solution

1. Step 1: Understand the pipeline concept

   Each stage in a pipeline processes data step-by-step, modifying or filtering it.

2. Step 2: Identify the role of stages

   Stages do not create databases or delete collections; they only transform or filter data.

3. Final Answer:

   Transforms or filters data before passing it to the next stage -> Option C

4. Quick Check:

   Pipeline stages transform or filter data [OK]

Hint: Remember: pipeline stages change data step-by-step [OK]

Common Mistakes:

• Thinking stages create or delete collections
• Confusing pipeline with database commands
• Assuming stages run SQL queries

2. Which of the following is the correct syntax to start an aggregation pipeline in MongoDB?

easy

A. db.collection.find({ $match: { status: 'A' } })

B. db.collection.update({ $match: { status: 'A' } })

C. db.collection.insert([{ $match: { status: 'A' } }])

D. db.collection.aggregate([ { $match: { status: 'A' } } ])

Solution

1. Step 1: Identify aggregation pipeline syntax

   The aggregation pipeline uses the aggregate() method with an array of stages.

2. Step 2: Compare options

   Only db.collection.aggregate([ { $match: { status: 'A' } } ]) uses aggregate() with a pipeline stage ($match). Others use find, insert, or update incorrectly.

3. Final Answer:

   db.collection.aggregate([ { $match: { status: 'A' } } ]) -> Option D

4. Quick Check:

   aggregate() starts pipelines [OK]

Hint: aggregate() method always starts a pipeline [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipelines
• Trying to insert pipeline stages
• Using update() with pipeline syntax

3. Given the pipeline: [{ $match: { age: { $gte: 18 } } }, { $group: { _id: '$city', count: { $sum: 1 } } }], what does the output represent?

medium

A. Count of people grouped by city who are 18 or older

B. List of all people younger than 18

C. Sum of ages for each city

D. All documents without filtering

Solution

1. Step 1: Analyze the $match stage

   This stage filters documents where age is 18 or more.

2. Step 2: Analyze the $group stage

   Groups filtered documents by city and counts how many per city.

3. Final Answer:

   Count of people grouped by city who are 18 or older -> Option A

4. Quick Check:

   $match filters, $group counts by city [OK]

Hint: $match filters, $group aggregates data [OK]

Common Mistakes:

• Confusing $sum with sum of ages
• Ignoring the filter condition
• Assuming output lists all people

4. What is wrong with this pipeline? [{ $match: { score: { $gt: 50 } } }, { $project: { score: 1, name: 1 } }, { $match: { name: 'Alice' } }]

medium

A. Using two $match stages is not allowed

B. The pipeline syntax is correct and will work as expected

C. The $project stage must include _id field explicitly

D. The second $match stage should come before $project

Solution

1. Step 1: Check multiple $match stages

   MongoDB allows multiple $match stages anywhere in the pipeline.

2. Step 2: Check $project usage

   $project can select fields; _id is included by default unless excluded explicitly.

3. Final Answer:

   The pipeline syntax is correct and will work as expected -> Option B

4. Quick Check:

   Multiple $match and $project usage is valid [OK]

Hint: Multiple $match stages are allowed anywhere [OK]

Common Mistakes:

• Thinking multiple $match stages are invalid
• Assuming _id must be included explicitly
• Believing $project must come last

5. You want to find the average price of products in each category but only for products with stock greater than 0. Which pipeline correctly achieves this?

hard

A. [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

B. [{ $group: { _id: '$category', avgPrice: { $avg: '$price' } } }, { $match: { stock: { $gt: 0 } } }]

C. [{ $project: { category: 1, price: 1, stock: 1 } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

D. [{ $match: { price: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$stock' } } }]

Solution

1. Step 1: Filter products with stock > 0 first

   The $match stage must come first to filter only products in stock.

2. Step 2: Group by category and calculate average price

   Use $group with _id as category and $avg on price to get average price per category.

3. Final Answer:

   [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }] -> Option A

4. Quick Check:

   Filter then group for correct average [OK]

Hint: Filter first, then group to aggregate correctly [OK]

Common Mistakes:

• Grouping before filtering causes wrong results
• Averaging wrong fields
• Filtering on price instead of stock