How the engine optimizes pipelines in MongoDB - Try It Yourself

Understanding How the MongoDB Engine Optimizes Pipelines

📖 Scenario: You are working with a MongoDB database that stores information about a bookstore's sales. You want to analyze sales data efficiently by using aggregation pipelines. MongoDB's engine optimizes these pipelines to run faster and use fewer resources.

🎯 Goal: Build a simple aggregation pipeline step-by-step to understand how MongoDB optimizes the stages for better performance.

📋 What You'll Learn

Create a collection called sales with sample documents representing book sales.

Add a configuration variable to filter sales by a minimum quantity.

Build an aggregation pipeline that filters and groups sales data.

Complete the pipeline by sorting the results to show the best-selling books first.

💡 Why This Matters

🌍 Real World

Aggregation pipelines are used in real-world applications to analyze and summarize large datasets efficiently, such as sales reports, user activity, or inventory management.

💼 Career

Understanding how to build and optimize MongoDB aggregation pipelines is a valuable skill for database developers, data analysts, and backend engineers working with NoSQL databases.

1

DATA SETUP: Create the sales collection with sample documents

Insert these exact documents into a MongoDB collection called sales: { book: "MongoDB Basics", quantity: 5, price: 20 }, { book: "Advanced MongoDB", quantity: 3, price: 35 }, { book: "MongoDB Basics", quantity: 2, price: 20 }, { book: "Data Science with MongoDB", quantity: 4, price: 40 }.

MongoDB

// Insert the exact documents into the sales collection
// Your code here

Hint

Use db.sales.insertMany([...]) with the exact documents listed.

2

CONFIGURATION: Define a minimum quantity filter variable

Create a variable called minQuantity and set it to 3 to filter sales with quantity greater or equal to this value.

MongoDB

db.sales.insertMany([
  { book: "MongoDB Basics", quantity: 5, price: 20 },
  { book: "Advanced MongoDB", quantity: 3, price: 35 },
  { book: "MongoDB Basics", quantity: 2, price: 20 },
  { book: "Data Science with MongoDB", quantity: 4, price: 40 }
])

// Create minQuantity variable and set it to 3
// Your code here

Hint

Use const minQuantity = 3 to define the filter variable.

3

CORE LOGIC: Build an aggregation pipeline to filter and group sales

Create a variable called pipeline that contains an aggregation pipeline array with these stages: a $match stage to filter documents where quantity is greater or equal to minQuantity, and a $group stage to group by book and sum the quantity as totalQuantity.

MongoDB

db.sales.insertMany([
  { book: "MongoDB Basics", quantity: 5, price: 20 },
  { book: "Advanced MongoDB", quantity: 3, price: 35 },
  { book: "MongoDB Basics", quantity: 2, price: 20 },
  { book: "Data Science with MongoDB", quantity: 4, price: 40 }
])

const minQuantity = 3

// Create the pipeline array with $match and $group stages
// Your code here

Hint

Use $match to filter and $group to sum quantities by book.

4

COMPLETION: Add a $sort stage to the pipeline to order by totalQuantity descending

Add a $sort stage to the pipeline array that sorts the results by totalQuantity in descending order.

MongoDB

db.sales.insertMany([
  { book: "MongoDB Basics", quantity: 5, price: 20 },
  { book: "Advanced MongoDB", quantity: 3, price: 35 },
  { book: "MongoDB Basics", quantity: 2, price: 20 },
  { book: "Data Science with MongoDB", quantity: 4, price: 40 }
])

const minQuantity = 3

const pipeline = [
  { $match: { quantity: { $gte: minQuantity } } },
  { $group: { _id: "$book", totalQuantity: { $sum: "$quantity" } } }
]

// Add a $sort stage to pipeline to sort by totalQuantity descending
// Your code here

Hint

Add { $sort: { totalQuantity: -1 } } as the last stage in the pipeline array.

Practice

1. What is the main reason MongoDB's aggregation engine reorders pipeline stages?

easy

A. To change the output format of the data

B. To add more fields to each document

C. To improve query performance by filtering data early

D. To sort the data alphabetically

Solution

1. Step 1: Understand pipeline stage purpose

   MongoDB pipelines process data step-by-step. Filtering early reduces data size quickly.

2. Step 2: Recognize optimization goal

   Reordering stages to filter first means less data passes through later stages, speeding up processing.

3. Final Answer:

   To improve query performance by filtering data early -> Option C

4. Quick Check:

   Filtering early = better performance [OK]

Hint: Filter data early in pipeline for faster queries [OK]

Common Mistakes:

• Thinking reordering changes output format
• Believing it adds fields automatically
• Assuming sorting is the main reason

2. Which of the following is the correct syntax to add a $match stage at the start of a MongoDB aggregation pipeline?

easy

A. [{$match: {status: "active"}}, {$group: {_id: "$category"}}]

B. [{$group: {_id: "$category"}}, {$match: {status: "active"}}]

C. [{$sort: {date: -1}}, {$match: {status: "active"}}]

D. [{$project: {name: 1}}, {$match: {status: "active"}}]

Solution

1. Step 1: Identify correct $match placement

   $match should come first to filter documents before grouping or sorting.

2. Step 2: Check syntax order

   [{$match: {status: "active"}}, {$group: {_id: "$category"}}] places $match first, then $group, which is correct syntax and optimization practice.

3. Final Answer:

   [{$match: {status: "active"}}, {$group: {_id: "$category"}}] -> Option A

4. Quick Check:

   $match first in pipeline = correct syntax [OK]

Hint: Place $match before $group to filter early [OK]

Common Mistakes:

• Putting $match after $group or $sort
• Using $match inside $group stage
• Incorrect array order of stages

3. Given the pipeline [{$project: {name: 1, age: 1}}, {$match: {age: {$gt: 30}}}], what will MongoDB do to optimize it?

medium

A. It will reorder to apply $match before $project

B. It will keep the stages as is, projecting before matching

C. It will remove the $project stage

D. It will combine $project and $match into one stage

Solution

1. Step 1: Analyze pipeline stages

   $project limits fields, $match filters documents. Filtering first reduces data early.

2. Step 2: Understand optimization behavior

   MongoDB reorders pipeline to apply $match before $project to reduce documents processed.

3. Final Answer:

   It will reorder to apply $match before $project -> Option A

4. Quick Check:

   Filter before project = better optimization [OK]

Hint: MongoDB moves $match before $project to filter early [OK]

Common Mistakes:

• Assuming stages stay in original order
• Thinking $project is removed
• Believing stages combine automatically

4. You wrote this pipeline: [{$group: {_id: "$category", total: {$sum: "$amount"}}}, {$match: {total: {$gt: 100}}}]. It runs slowly. What is the main issue?

medium

A. You cannot use $sum inside $group

B. The $group stage syntax is incorrect

C. The pipeline is missing a $sort stage

D. The $match stage should come before $group to filter documents early

Solution

1. Step 1: Understand pipeline order impact

   $group aggregates all documents first, then $match filters groups, causing more work.

2. Step 2: Identify optimization fix

   Filtering documents before grouping reduces data size and speeds up aggregation.

3. Final Answer:

   The $match stage should come before $group to filter documents early -> Option D

4. Quick Check:

   Filter before group = faster pipeline [OK]

Hint: Place $match before $group to reduce data early [OK]

Common Mistakes:

• Thinking $group syntax is wrong
• Believing $sum can't be used in $group
• Assuming $sort affects speed here

5. You want to optimize this pipeline for a large collection:
[{$lookup: {from: "orders", localField: "_id", foreignField: "userId", as: "orders"}}, {$unwind: "$orders"}, {$match: {"orders.status": "shipped"}}, {$group: {_id: "$_id", total: {$sum: "$orders.amount"}}}]
What is the best optimization to improve performance?

hard

A. Replace $lookup with $project

B. Move the $match stage before $lookup using a $match on the local collection

C. Remove the $unwind stage

D. Add a $sort stage before $group

Solution

1. Step 1: Analyze pipeline stages

   $lookup joins data, $unwind expands arrays, $match filters joined data, $group aggregates.

2. Step 2: Identify optimization strategy

   Filtering local documents before $lookup reduces join size, improving performance.

3. Step 3: Understand why other options fail

   $sort doesn't reduce data size, removing $unwind breaks structure, $project can't replace $lookup.

4. Final Answer:

   Move the $match stage before $lookup using a $match on the local collection -> Option B

5. Quick Check:

   Filter early before $lookup = best optimization [OK]

Hint: Filter local data before $lookup to speed joins [OK]

Common Mistakes:

• Adding $sort expecting speedup
• Removing $unwind breaks data structure
• Replacing $lookup with $project is invalid