$out and $merge for writing results in MongoDB - Step-by-Step Execution

Concept Flow - $out and $merge for writing results

Start Aggregation Pipeline

↓

Process Documents

↓

$out

↓

Replace Target

↓

Write Complete

↓

End

The pipeline processes documents, then either replaces a collection with $out or merges results into a collection with $merge.

Execution Sample

MongoDB

db.sales.aggregate([
  { $match: { status: "A" } },
  { $group: { _id: "$item", total: { $sum: "$amount" } } },
  { $merge: { into: "totals", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
])

This pipeline filters sales with status 'A', groups by item summing amounts, then merges results into 'totals' collection.

Execution Table

Step	Action	Documents Processed	Write Stage	Target Collection	Result
1	Match status 'A'	Filters documents with status 'A'	N/A	N/A	Filtered documents passed on
2	Group by item	Groups filtered docs by item, sums amounts	N/A	N/A	Grouped documents produced
3	Write with $merge	Uses grouped docs	$merge	totals	Documents merged into 'totals' collection
4	End	No more stages	N/A	N/A	Aggregation complete

💡 Pipeline ends after writing results with $merge to 'totals' collection

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3
Documents	All sales docs	Filtered docs with status 'A'	Grouped docs by item with totals	Written to 'totals' collection

Key Moments - 2 Insights

Why does $out replace the entire target collection but $merge can update or insert documents?

$out replaces the whole target collection with the pipeline output (see execution_table step 3 for $merge behavior). $merge allows more control by matching documents and updating or inserting them, preserving existing data not matched.

What happens if the target collection does not exist when using $merge?

If the target collection doesn't exist, $merge creates it automatically before inserting documents, as shown in execution_table step 3.

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what write stage is used to update the 'totals' collection?

A $merge

B $out

C $group

D $match

💡 Hint

Check the 'Write Stage' column in execution_table row for step 3

According to variable_tracker, what is the state of 'Documents' after step 2?

A All sales documents

B Filtered docs with status 'A'

C Grouped docs by item with totals

D Written to 'totals' collection

💡 Hint

Look at the 'After Step 2' column for 'Documents' in variable_tracker

If we replace $merge with $out in the sample code, what would happen to the 'totals' collection?

A It would be merged with existing documents

B It would be replaced entirely by the pipeline output

C It would remain unchanged

D It would cause an error

💡 Hint

Recall the concept_flow difference between $out and $merge stages

Concept Snapshot

$out and $merge write pipeline results to collections.
$out replaces the entire target collection.
$merge updates or inserts documents based on matching keys.
Use $merge for incremental updates, $out for full replacement.
Both create target collection if missing.

Full Transcript

This visual execution shows how MongoDB aggregation pipelines can write results using $out or $merge stages. The pipeline starts by processing documents, such as filtering and grouping. Then, it chooses a write stage. The $out stage replaces the entire target collection with the pipeline output, effectively overwriting it. The $merge stage merges the pipeline output into the target collection by matching documents on specified keys, updating existing documents or inserting new ones. The execution table traces each step, showing document states and write actions. The variable tracker follows how documents change through the pipeline. Key moments clarify common confusions about $out replacing collections and $merge updating them. The quiz tests understanding of these behaviors referencing the execution visuals. The snapshot summarizes the key differences and usage of $out and $merge for writing aggregation results.

Practice

1. What is the main difference between the $out and $merge stages in MongoDB aggregation?

easy

A. $out updates documents, and $merge deletes the target collection.

B. $out only works with sharded clusters, $merge does not.

C. $out replaces the entire target collection, while $merge updates or inserts documents based on matching keys.

D. $out appends documents without replacing, $merge replaces the collection.

Solution

1. Step 1: Understand $out behavior

   $out writes aggregation results by replacing the entire target collection with new data.

2. Step 2: Understand $merge behavior

   $merge writes results by updating existing documents or inserting new ones based on matching keys, preserving unmatched documents.

3. Final Answer:

   $out replaces collection; $merge updates/inserts by keys -> Option C

4. Quick Check:

   $out replaces, $merge merges [OK]

Hint: Remember: $out replaces, $merge merges by keys [OK]

Common Mistakes:

• Thinking $out appends data instead of replacing
• Confusing $merge with deleting collections
• Assuming $merge always replaces entire collection

2. Which of the following is the correct syntax to use $out in a MongoDB aggregation pipeline?

easy

A. { $out: "targetCollection" }

B. { $out: { collection: "targetCollection" } }

C. { $out: ["targetCollection"] }

D. { $out: { db: "targetDB" } }

Solution

1. Step 1: Recall $out syntax

   The $out stage requires the target collection name as a string directly, like { $out: "collectionName" }.

2. Step 2: Check options

   { $out: "targetCollection" } matches the correct syntax. The other options use incorrect formats such as extra objects or arrays.

3. Final Answer:

   { $out: "targetCollection" } -> Option A

4. Quick Check:

   $out needs collection name string [OK]

Hint: Use simple string for collection name in $out [OK]

Common Mistakes:

• Using object or array instead of string for collection name
• Trying to specify database inside $out stage
• Adding extra braces or brackets around collection name

3. Given the aggregation pipeline below, what will be the result in the summary collection after running it?

db.sales.aggregate([
  { $group: { _id: "$category", total: { $sum: "$amount" } } },
  { $merge: { into: "summary", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
])

medium

A. The summary collection will remain unchanged.

B. The summary collection will be completely replaced with the aggregation results.

C. The aggregation will fail due to missing $out stage.

D. The summary collection will contain documents with total sales per category, replacing existing matching documents.

Solution

1. Step 1: Understand the $group stage

   This groups sales by category and sums amounts, producing documents like { _id: category, total: sum }.

2. Step 2: Understand the $merge stage

   $merge writes results into summary collection, replacing documents with matching _id or inserting new ones.

3. Final Answer:

   summary collection updated with total sales per category, replacing matches -> Option D

4. Quick Check:

   $merge updates or inserts by key [OK]

Hint: Remember $merge updates matching keys, inserts new [OK]

Common Mistakes:

• Thinking $merge replaces entire collection like $out
• Assuming aggregation fails without $out
• Believing collection stays unchanged after $merge

4. You run this aggregation pipeline but get an error:

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $out: { collection: "shippedOrders" } }
])

What is the likely cause of the error?

medium

A. You cannot use $out after $match stage.

B. The $out stage syntax is incorrect; it should be a string, not an object.

C. The $match stage is invalid and causes the error.

D. The target collection shippedOrders does not exist.

Solution

1. Step 1: Check $out syntax

   $out requires the target collection name as a string, not an object with a collection field.

2. Step 2: Analyze error cause

   Using { collection: "shippedOrders" } causes syntax error; correct usage is { $out: "shippedOrders" }.

3. Final Answer:

   Incorrect $out syntax using object instead of string -> Option B

4. Quick Check:

   $out needs string collection name [OK]

Hint: Use string for $out target, not object [OK]

Common Mistakes:

• Assuming $match causes error
• Thinking $out requires existing collection
• Believing $out cannot follow $match

5. You want to aggregate user activity data and save results into a collection named userStats. You want to update existing documents by userId and insert new ones if missing, but keep other fields unchanged when matched. Which $merge option should you use?

hard

A. { into: "userStats", on: "userId", whenMatched: "merge", whenNotMatched: "insert" }

B. { into: "userStats", on: "userId", whenMatched: "replace", whenNotMatched: "insert" }

C. { into: "userStats", on: "userId", whenMatched: "fail", whenNotMatched: "insert" }

D. { into: "userStats", on: "userId", whenMatched: "keepExisting", whenNotMatched: "discard" }

Solution

1. Step 1: Understand whenMatched options

   whenMatched: "merge" updates only specified fields, keeping others unchanged; replace overwrites entire document.

2. Step 2: Choose correct $merge options

   To update existing documents by userId and insert new ones, use whenMatched: "merge" and whenNotMatched: "insert".

3. Final Answer:

   { into: "userStats", on: "userId", whenMatched: "merge", whenNotMatched: "insert" } -> Option A

4. Quick Check:

   whenMatched: "merge" updates fields, inserts new [OK]

Hint: Use whenMatched: "merge" to update fields, not replace [OK]

Common Mistakes:

• Using replace which overwrites entire documents
• Choosing fail which errors on matches
• Using invalid whenNotMatched options