Why advanced stages matter in MongoDB - Performance Analysis

Time Complexity: Why advanced stages matter

O(n log n)

Understanding Time Complexity

When using MongoDB's aggregation, each stage processes data and passes results to the next. Understanding how time grows with each stage helps us see why advanced stages matter.

We want to know how adding more stages affects the total work done.

Scenario Under Consideration

Analyze the time complexity of this aggregation pipeline with multiple stages.

db.collection.aggregate([
  { $match: { status: "active" } },
  { $group: { _id: "$category", total: { $sum: "$amount" } } },
  { $sort: { total: -1 } },
  { $limit: 5 }
])
    

This pipeline filters documents, groups them by category summing amounts, sorts by total, and limits output to top 5.

Identify Repeating Operations

Look at what repeats as data flows through stages.

• Primary operation: Scanning documents in $match and grouping in $group.
• How many times: Each document is processed once per stage until filtered out.

How Execution Grows With Input

As the number of documents grows, each stage processes more data, increasing total work.

Input Size (n)	Approx. Operations
10	About 10 operations per stage
100	About 100 operations per stage
1000	About 1000 operations per stage

Pattern observation: Work grows roughly in direct proportion to input size for each stage.

Final Time Complexity

Time Complexity: O(n log n)

This means the total work grows roughly in proportion to n log n due to the $sort stage.

Common Mistake

[X] Wrong: "Adding more stages doesn't affect performance much because each stage only does a small task."

[OK] Correct: Each stage processes data, so more stages mean more passes over data, increasing total work.

Interview Connect

Understanding how each stage adds to total work helps you explain real MongoDB queries clearly and shows you think about efficiency in data processing.

Self-Check

"What if we added an extra $project stage before $group? How would that change the time complexity?"

Practice

1. What is the main reason to use advanced stages in MongoDB aggregation pipelines?

easy

A. To process data step-by-step for clearer results

B. To store data in multiple collections

C. To create indexes automatically

D. To backup the database

Solution

1. Step 1: Understand the purpose of advanced stages

   Advanced stages in MongoDB aggregation allow processing data in steps like filtering, grouping, and sorting.

2. Step 2: Identify the benefit of step-by-step processing

   This stepwise processing helps produce clear and useful results from complex data.

3. Final Answer:

   To process data step-by-step for clearer results -> Option A

4. Quick Check:

   Advanced stages = step-by-step processing [OK]

Hint: Think of stages as steps to clean and organize data [OK]

Common Mistakes:

• Confusing aggregation stages with indexing
• Thinking advanced stages create backups
• Believing stages store data in collections

2. Which of the following is the correct syntax to start an aggregation pipeline in MongoDB?

easy

A. db.collection.match([{ age: { $gt: 25 } }])

B. db.collection.find({ $match: { age: { $gt: 25 } } })

C. db.collection.aggregate([{ $match: { age: { $gt: 25 } } }])

D. db.collection.aggregate({ $match: { age: { $gt: 25 } } })

Solution

1. Step 1: Recall the aggregation method syntax

   The aggregation pipeline starts with db.collection.aggregate() and takes an array of stages.

2. Step 2: Check the correct use of $match stage inside an array

   The $match stage must be inside an array as the first element to filter documents.

3. Final Answer:

   db.collection.aggregate([{ $match: { age: { $gt: 25 } } }]) -> Option C

4. Quick Check:

   aggregate() takes an array of stages [OK]

Hint: Remember aggregate() needs an array of stages [OK]

Common Mistakes:

• Using find() instead of aggregate() for pipeline
• Passing object instead of array to aggregate()
• Using non-existent match() method

3. Given the collection 'sales' with documents { item: 'apple', qty: 5 } and { item: 'banana', qty: 10 }, what is the output of this pipeline?

db.sales.aggregate([
  { $match: { qty: { $gt: 5 } } },
  { $project: { item: 1, _id: 0 } }
])

medium

A. [{ item: 'apple' }]

B. [{ item: 'banana' }]

C. [{ item: 'apple' }, { item: 'banana' }]

D. []

Solution

1. Step 1: Apply the $match stage filtering qty > 5

   Only the document with qty 10 (banana) matches the condition.

2. Step 2: Apply the $project stage to include item and exclude _id

   The output will show only the item field for the matched document.

3. Final Answer:

   [{ item: 'banana' }] -> Option B

4. Quick Check:

   Filter qty > 5 then project item only [OK]

Hint: Filter first, then project fields you want [OK]

Common Mistakes:

• Including documents with qty <= 5
• Forgetting to exclude _id field
• Projecting fields before filtering

4. Identify the error in this aggregation pipeline:

db.orders.aggregate([
  { $group: { _id: '$customerId', total: { $sum: '$amount' } } },
  { $sort: { total: -1 } },
  { $match: { total: { $gt: 100 } } }
])

medium

A. The $sort stage cannot be used after $group

B. The $match stage should come before $group

C. The $match stage cannot filter on computed fields after $group

D. No error, pipeline is correct

Solution

1. Step 1: Analyze the $group stage

   It correctly groups by customerId and computes the total sum per customer.

2. Step 2: Verify $sort and $match stages

   $sort by total descending after $group is valid. $match filtering groups with total > 100 after $group is also valid on computed fields.

3. Final Answer:

   No error, pipeline is correct -> Option D

4. Quick Check:

   $match after $group on aggregates is allowed [OK]

Hint: $match can filter aggregates after $group [OK]

Common Mistakes:

• Thinking $match must always precede $group
• Believing $sort cannot follow $group
• Assuming $match cannot filter computed fields

5. You want to find the top 3 products by total sales amount from a 'sales' collection. Which pipeline correctly uses advanced stages to achieve this?

hard

A. [{ $group: { _id: '$product', totalSales: { $sum: '$amount' } } }, { $sort: { totalSales: -1 } }, { $limit: 3 }]

B. [{ $sort: { amount: -1 } }, { $limit: 3 }, { $group: { _id: '$product', totalSales: { $sum: '$amount' } } }]

C. [{ $limit: 3 }, { $group: { _id: '$product', totalSales: { $sum: '$amount' } } }, { $sort: { totalSales: -1 } }]

D. [{ $group: { _id: '$product', totalSales: { $sum: '$amount' } } }, { $limit: 3 }, { $sort: { totalSales: -1 } }]

Solution

1. Step 1: Group sales by product and sum amounts

   The $group stage aggregates total sales per product correctly.

2. Step 2: Sort products by totalSales descending and limit to top 3

   $sort orders by totalSales descending, then $limit picks top 3 products.

3. Final Answer:

   [{ $group: { _id: '$product', totalSales: { $sum: '$amount' } } }, { $sort: { totalSales: -1 } }, { $limit: 3 }] -> Option A

4. Quick Check:

   Group, then sort, then limit for top results [OK]

Hint: Group first, then sort and limit for top results [OK]

Common Mistakes:

• Limiting before grouping loses data
• Sorting before grouping gives wrong totals
• Placing $limit before $sort causes wrong output