Pipeline execution order matters in MongoDB - Step-by-Step Execution

Concept Flow - Pipeline execution order matters

Start with collection

↓

Apply first pipeline stage

↓

Transform documents

↓

Apply second pipeline stage

↓

Transform documents

↓

Final output after last stage

Documents flow through each pipeline stage in order; changing the order changes the final result.

Execution Sample

MongoDB

db.sales.aggregate([
  { $match: { status: "A" } },
  { $group: { _id: "$cust_id", total: { $sum: "$amount" } } }
])

Filters sales with status 'A' first, then groups by customer to sum amounts.

Execution Table

Step	Pipeline Stage	Input Documents	Action	Output Documents
1	{ $match: { status: "A" } }	[{cust_id:1, status:"A", amount:100}, {cust_id:2, status:"B", amount:200}, {cust_id:1, status:"A", amount:50}]	Filter documents where status is 'A'	[{cust_id:1, status:"A", amount:100}, {cust_id:1, status:"A", amount:50}]
2	{ $group: { _id: "$cust_id", total: { $sum: "$amount" } } }	[{cust_id:1, status:"A", amount:100}, {cust_id:1, status:"A", amount:50}]	Group by cust_id and sum amounts	[{_id:1, total:150}]
3	End of pipeline	[{_id:1, total:150}]	Return final documents	[{_id:1, total:150}]

💡 Pipeline ends after last stage; documents transformed step-by-step.

Variable Tracker

Variable	Start	After Step 1	After Step 2	Final
Documents	[{cust_id:1, status:"A", amount:100}, {cust_id:2, status:"B", amount:200}, {cust_id:1, status:"A", amount:50}]	[{cust_id:1, status:"A", amount:100}, {cust_id:1, status:"A", amount:50}]	[{_id:1, total:150}]	[{_id:1, total:150}]

Key Moments - 2 Insights

Why does changing the order of $match and $group change the result?

Because $match filters documents first, reducing data for $group. If $group runs first, it groups all documents, including unwanted ones, changing totals. See execution_table rows 1 and 2.

What happens if $group is placed before $match in this example?

Grouping happens on all documents, including those with status 'B', so totals include unwanted data. This differs from filtering first as shown in execution_table row 1.

Visual Quiz - 3 Questions

Look at the execution_table, what is the output after the first pipeline stage?

A [{_id:1, total:150}]

B [{cust_id:2, status:"B", amount:200}]

C [{cust_id:1, status:"A", amount:100}, {cust_id:1, status:"A", amount:50}]

D []

💡 Hint

Check the Output Documents column in row 1 of execution_table.

At which step does the documents get grouped by cust_id?

A Step 2

B Step 1

C Step 3

D No grouping occurs

💡 Hint

Look at the Pipeline Stage column and Action in execution_table row 2.

If we swap $group and $match stages, what changes in the output?

A Output remains the same

B Grouping includes documents with status 'B'

C No documents are returned

D Pipeline fails with error

💡 Hint

Refer to key_moments explanation about stage order impact.

Concept Snapshot

MongoDB aggregation pipelines process documents stage by stage.
Order matters: earlier stages filter or transform data for later stages.
$match filters documents; $group aggregates them.
Changing order changes results.
Always filter early to optimize and get correct output.

Full Transcript

In MongoDB aggregation pipelines, documents flow through each stage in the order written. Each stage transforms the documents before passing them on. For example, applying $match first filters documents, so the next $group stage only processes filtered data. If you reverse the order, $group aggregates all documents, including unwanted ones, changing the final result. This visual trace shows how documents change after each stage, highlighting why pipeline order matters.

Practice

1. In MongoDB aggregation pipelines, why does the order of stages matter?

easy

A. Because stages run in the order they are written, affecting results and performance

B. Because MongoDB runs all stages in parallel regardless of order

C. Because only the last stage affects the output

D. Because stages can be reordered automatically by MongoDB

Solution

1. Step 1: Understand pipeline execution

   MongoDB executes pipeline stages sequentially in the order they appear.

2. Step 2: Impact of order on results and performance

   Changing the order can change the output and how fast the query runs, especially filtering early helps performance.

3. Final Answer:

   Because stages run in the order they are written, affecting results and performance -> Option A

4. Quick Check:

   Pipeline order affects output and speed [OK]

Hint: Remember: pipeline stages run top to bottom [OK]

Common Mistakes:

• Thinking MongoDB runs stages in parallel
• Believing order does not affect output
• Assuming MongoDB reorders stages automatically

2. Which of the following is the correct syntax to start a MongoDB aggregation pipeline with a filter stage?

easy

A. db.collection.aggregate([{ $match: { status: 'A' } }])

B. db.collection.aggregate({ $match: { status: 'A' } })

C. db.collection.find({ $match: { status: 'A' } })

D. db.collection.aggregate([{ match: { status: 'A' } }])

Solution

1. Step 1: Identify correct aggregation syntax

   The aggregate method takes an array of stages, each stage is an object with a stage operator like $match.

2. Step 2: Check each option

   db.collection.aggregate([{ $match: { status: 'A' } }]) correctly uses an array with $match stage. db.collection.aggregate({ $match: { status: 'A' } }) misses array brackets. db.collection.find({ $match: { status: 'A' } }) uses find() which is not aggregation. db.collection.aggregate([{ match: { status: 'A' } }]) misses $ in $match.

3. Final Answer:

   db.collection.aggregate([{ $match: { status: 'A' } }]) -> Option A

4. Quick Check:

   Aggregation stages are in an array [OK]

Hint: Aggregation stages must be in an array [OK]

Common Mistakes:

• Omitting the array brackets around stages
• Using find() instead of aggregate() for pipelines
• Missing $ in stage operators like $match

3. Given this pipeline:

db.orders.aggregate([
  { $sort: { price: -1 } },
  { $match: { status: 'A' } }
])

What is the main issue with this pipeline?

medium

A. It sorts ascending instead of descending

B. It will cause a syntax error due to stage order

C. It will return no results because $match must come first

D. It filters after sorting, which is less efficient

Solution

1. Step 1: Analyze stage order

   The pipeline sorts all documents first, then filters with $match.

2. Step 2: Understand performance impact

   Filtering first reduces documents early, making sorting faster. Sorting before filtering wastes resources.

3. Final Answer:

   It filters after sorting, which is less efficient -> Option D

4. Quick Check:

   Filter early, then sort for best performance [OK]

Hint: Filter before sort to improve speed [OK]

Common Mistakes:

• Thinking stage order causes syntax errors
• Believing $match must always be first to work
• Confusing sort order direction

4. You wrote this pipeline but it returns an error:

db.sales.aggregate([
  { $group: { _id: "$item", total: { $sum: "$amount" } } },
  { $match: { $gt: ["$total", 100] } }
])

What is the likely cause?

medium

A. The $match stage cannot filter on fields created by $group

B. The $match stage should use $expr to filter on computed fields

C. The $match stage must come before $group to filter correctly

D. The $group stage syntax is incorrect

Solution

1. Step 1: Identify the invalid $match syntax

   The $match stage uses aggregation expression syntax { $gt: ["$total", 100] } directly, which is not valid query selector syntax.

2. Step 2: Use $expr for aggregation expressions

   To use expression operators like $gt with field paths and literals in array form, wrap in $expr: { $match: { $expr: { $gt: ["$total", 100] } } }.

3. Final Answer:

   The $match stage should use $expr to filter on computed fields -> Option B

4. Quick Check:

   $expr enables agg expressions in $match [OK]

Hint: Use $expr in $match after $group [OK]

Common Mistakes:

• Trying to filter grouped fields without $expr
• Placing $match before $group expecting same results
• Assuming $group syntax is wrong

5. You want to find the top 3 most expensive products with status 'A'. Which pipeline correctly applies the stages for best performance?

hard

A. db.products.aggregate([
{ $sort: { price: -1 } },
{ $limit: 3 },
{ $match: { status: 'A' } }
])

B. db.products.aggregate([
{ $limit: 3 },
{ $match: { status: 'A' } },
{ $sort: { price: -1 } }
])

C. db.products.aggregate([
{ $match: { status: 'A' } },
{ $sort: { price: -1 } },
{ $limit: 3 }
])

D. db.products.aggregate([
{ $match: { status: 'A' } },
{ $limit: 3 },
{ $sort: { price: -1 } }
])

Solution

1. Step 1: Filter early for performance

   Applying $match first reduces documents to only those with status 'A'.

2. Step 2: Sort and limit after filtering

   Sorting after filtering is faster, then $limit picks top 3.

3. Step 3: Check other options

   db.products.aggregate([ { $sort: { price: -1 } }, { $limit: 3 }, { $match: { status: 'A' } } ]) sorts all documents before filtering, wasting resources. db.products.aggregate([ { $limit: 3 }, { $match: { status: 'A' } }, { $sort: { price: -1 } } ]) limits before filtering, wrong results. db.products.aggregate([ { $match: { status: 'A' } }, { $limit: 3 }, { $sort: { price: -1 } } ]) limits before sorting, wrong order.

4. Final Answer:

   db.products.aggregate([ { $match: { status: 'A' } }, { $sort: { price: -1 } }, { $limit: 3 } ]) -> Option C

5. Quick Check:

   Filter, then sort, then limit [OK]

Hint: Filter first, then sort, then limit [OK]

Common Mistakes:

• Sorting before filtering wastes resources
• Limiting before sorting gives wrong top results
• Filtering after limiting misses correct documents