Pipeline mental model (stages flow) in MongoDB - Step-by-Step Execution

Concept Flow - Pipeline mental model (stages flow)

Input Documents

↓

Stage 1: $match

↓

Stage 2: $group

↓

Stage 3: $sort

↓

Stage 4: $project

↓

Output Documents

Documents flow through each pipeline stage in order, transforming data step-by-step until the final output is produced.

Execution Sample

MongoDB

db.sales.aggregate([
  { $match: { status: "A" } },
  { $group: { _id: "$cust_id", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $project: { total: 1, _id: 0 } }
])

This pipeline filters sales with status 'A', groups by customer ID summing amounts, sorts by total descending, then projects only the total.

Execution Table

Step	Stage	Input Documents	Action	Output Documents
1	$match	[{cust_id:1, status:'A', amount:100}, {cust_id:2, status:'B', amount:200}, {cust_id:1, status:'A', amount:50}]	Filter documents where status = 'A'	[{cust_id:1, status:'A', amount:100}, {cust_id:1, status:'A', amount:50}]
2	$group	[{cust_id:1, status:'A', amount:100}, {cust_id:1, status:'A', amount:50}]	Group by cust_id and sum amounts	[{_id:1, total:150}]
3	$sort	[{_id:1, total:150}]	Sort by total descending (only one document, no change)	[{_id:1, total:150}]
4	$project	[{_id:1, total:150}]	Include only total field, exclude _id	[{total:150}]
5	End	[{total:150}]	Pipeline complete, output final documents	[{total:150}]

💡 All stages processed, pipeline ends with final transformed documents.

Variable Tracker

Variable	Start	After 1	After 2	After 3	After 4	Final
documents	[{cust_id:1, status:'A', amount:100}, {cust_id:2, status:'B', amount:200}, {cust_id:1, status:'A', amount:50}]	[{cust_id:1, status:'A', amount:100}, {cust_id:1, status:'A', amount:50}]	[{_id:1, total:150}]	[{_id:1, total:150}]	[{total:150}]	[{total:150}]

Key Moments - 3 Insights

Why does the $match stage reduce the number of documents?

Because $match filters documents based on a condition (status = 'A'), only documents meeting this condition pass to the next stage, as shown in execution_table row 1.

How does $group change the shape of documents?

$group aggregates documents by a key (cust_id) and computes sums, producing new documents with _id and total fields, replacing original fields (see execution_table row 2).

Why does $project exclude the _id field in the final output?

$project controls which fields appear in output documents; setting _id:0 removes it, so only total remains, as shown in execution_table row 4.

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what is the total amount for cust_id 1?

A 150

B 100

C 50

D 200

💡 Hint

Check the output documents column at step 2 in the execution_table.

At which step does the number of documents reduce from 3 to 2?

A Step 3 ($sort)

B Step 2 ($group)

C Step 1 ($match)

D Step 4 ($project)

💡 Hint

Look at the Input and Output Documents columns in execution_table rows 1 and 2.

If we remove the $project stage, what would the final output documents include?

A Only total field

B Both _id and total fields

C Only _id field

D No fields

💡 Hint

Compare output documents at steps 4 and 5 in execution_table.

Concept Snapshot

MongoDB aggregation pipeline processes documents through ordered stages.
Each stage transforms documents (filter, group, sort, project).
Documents flow from input to output step-by-step.
Stages like $match filter, $group aggregates, $sort orders, $project shapes output.
Final output is the result after all stages run.

Full Transcript

The MongoDB aggregation pipeline works by passing documents through a series of stages. Each stage takes the documents from the previous stage, applies a transformation, and passes the results to the next stage. For example, $match filters documents by a condition, reducing the number of documents. Then $group aggregates documents by a key and computes sums, changing the document structure. $sort orders the documents, and $project selects which fields to keep in the output. This flow continues until all stages are processed, producing the final output documents. The execution table shows each step's input, action, and output, helping visualize how documents change through the pipeline.

Practice

1. What does each stage in a MongoDB aggregation pipeline do?

easy

A. Runs a SQL query on the data

B. Creates a new database

C. Transforms or filters data before passing it to the next stage

D. Deletes the entire collection

Solution

1. Step 1: Understand the pipeline concept

   Each stage in a pipeline processes data step-by-step, modifying or filtering it.

2. Step 2: Identify the role of stages

   Stages do not create databases or delete collections; they only transform or filter data.

3. Final Answer:

   Transforms or filters data before passing it to the next stage -> Option C

4. Quick Check:

   Pipeline stages transform or filter data [OK]

Hint: Remember: pipeline stages change data step-by-step [OK]

Common Mistakes:

• Thinking stages create or delete collections
• Confusing pipeline with database commands
• Assuming stages run SQL queries

2. Which of the following is the correct syntax to start an aggregation pipeline in MongoDB?

easy

A. db.collection.find({ $match: { status: 'A' } })

B. db.collection.update({ $match: { status: 'A' } })

C. db.collection.insert([{ $match: { status: 'A' } }])

D. db.collection.aggregate([ { $match: { status: 'A' } } ])

Solution

1. Step 1: Identify aggregation pipeline syntax

   The aggregation pipeline uses the aggregate() method with an array of stages.

2. Step 2: Compare options

   Only db.collection.aggregate([ { $match: { status: 'A' } } ]) uses aggregate() with a pipeline stage ($match). Others use find, insert, or update incorrectly.

3. Final Answer:

   db.collection.aggregate([ { $match: { status: 'A' } } ]) -> Option D

4. Quick Check:

   aggregate() starts pipelines [OK]

Hint: aggregate() method always starts a pipeline [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipelines
• Trying to insert pipeline stages
• Using update() with pipeline syntax

3. Given the pipeline: [{ $match: { age: { $gte: 18 } } }, { $group: { _id: '$city', count: { $sum: 1 } } }], what does the output represent?

medium

A. Count of people grouped by city who are 18 or older

B. List of all people younger than 18

C. Sum of ages for each city

D. All documents without filtering

Solution

1. Step 1: Analyze the $match stage

   This stage filters documents where age is 18 or more.

2. Step 2: Analyze the $group stage

   Groups filtered documents by city and counts how many per city.

3. Final Answer:

   Count of people grouped by city who are 18 or older -> Option A

4. Quick Check:

   $match filters, $group counts by city [OK]

Hint: $match filters, $group aggregates data [OK]

Common Mistakes:

• Confusing $sum with sum of ages
• Ignoring the filter condition
• Assuming output lists all people

4. What is wrong with this pipeline? [{ $match: { score: { $gt: 50 } } }, { $project: { score: 1, name: 1 } }, { $match: { name: 'Alice' } }]

medium

A. Using two $match stages is not allowed

B. The pipeline syntax is correct and will work as expected

C. The $project stage must include _id field explicitly

D. The second $match stage should come before $project

Solution

1. Step 1: Check multiple $match stages

   MongoDB allows multiple $match stages anywhere in the pipeline.

2. Step 2: Check $project usage

   $project can select fields; _id is included by default unless excluded explicitly.

3. Final Answer:

   The pipeline syntax is correct and will work as expected -> Option B

4. Quick Check:

   Multiple $match and $project usage is valid [OK]

Hint: Multiple $match stages are allowed anywhere [OK]

Common Mistakes:

• Thinking multiple $match stages are invalid
• Assuming _id must be included explicitly
• Believing $project must come last

5. You want to find the average price of products in each category but only for products with stock greater than 0. Which pipeline correctly achieves this?

hard

A. [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

B. [{ $group: { _id: '$category', avgPrice: { $avg: '$price' } } }, { $match: { stock: { $gt: 0 } } }]

C. [{ $project: { category: 1, price: 1, stock: 1 } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }]

D. [{ $match: { price: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$stock' } } }]

Solution

1. Step 1: Filter products with stock > 0 first

   The $match stage must come first to filter only products in stock.

2. Step 2: Group by category and calculate average price

   Use $group with _id as category and $avg on price to get average price per category.

3. Final Answer:

   [{ $match: { stock: { $gt: 0 } } }, { $group: { _id: '$category', avgPrice: { $avg: '$price' } } }] -> Option A

4. Quick Check:

   Filter then group for correct average [OK]

Hint: Filter first, then group to aggregate correctly [OK]

Common Mistakes:

• Grouping before filtering causes wrong results
• Averaging wrong fields
• Filtering on price instead of stock