Why $out and $merge for writing results in MongoDB? - Purpose & Use Cases

The Big Idea

What if you could save your filtered data automatically without lifting a finger?

The Scenario

Imagine you have a huge list of customer orders in one notebook, and you want to create a new notebook that only has orders from last month. Doing this by hand means copying each order one by one, which takes forever and can easily lead to mistakes.

The Problem

Manually filtering and copying data is slow and tiring. You might miss some orders or copy wrong details. Also, if the original list changes, you have to repeat the whole process again, wasting time and risking errors.

The Solution

Using $out and $merge in MongoDB lets you automatically save the results of your data filtering or transformation into a new or existing collection. This means you don't have to copy anything by hand, and your data stays accurate and up to date.

Before vs After

✗ Before

Find orders from last month, write them down in a new notebook by hand.

✓ After

db.orders.aggregate([{ $match: { date: { $gte: start, $lt: end } } }, { $out: 'lastMonthOrders' }])

What It Enables

You can quickly create or update collections with filtered or transformed data, making your database smarter and your work easier.

Real Life Example

A store manager wants to analyze only last month's sales. Using $out, they create a new collection with just those sales, ready for fast reports without touching the original data.

Key Takeaways

Manually copying filtered data is slow and error-prone.

$out and $merge automate saving query results into collections.

This keeps data organized, accurate, and easy to update.

Practice

1. What is the main difference between the $out and $merge stages in MongoDB aggregation?

easy

A. $out updates documents, and $merge deletes the target collection.

B. $out only works with sharded clusters, $merge does not.

C. $out replaces the entire target collection, while $merge updates or inserts documents based on matching keys.

D. $out appends documents without replacing, $merge replaces the collection.

Solution

1. Step 1: Understand $out behavior

   $out writes aggregation results by replacing the entire target collection with new data.

2. Step 2: Understand $merge behavior

   $merge writes results by updating existing documents or inserting new ones based on matching keys, preserving unmatched documents.

3. Final Answer:

   $out replaces collection; $merge updates/inserts by keys -> Option C

4. Quick Check:

   $out replaces, $merge merges [OK]

Hint: Remember: $out replaces, $merge merges by keys [OK]

Common Mistakes:

• Thinking $out appends data instead of replacing
• Confusing $merge with deleting collections
• Assuming $merge always replaces entire collection

2. Which of the following is the correct syntax to use $out in a MongoDB aggregation pipeline?

easy

A. { $out: "targetCollection" }

B. { $out: { collection: "targetCollection" } }

C. { $out: ["targetCollection"] }

D. { $out: { db: "targetDB" } }

Solution

1. Step 1: Recall $out syntax

   The $out stage requires the target collection name as a string directly, like { $out: "collectionName" }.

2. Step 2: Check options

   { $out: "targetCollection" } matches the correct syntax. The other options use incorrect formats such as extra objects or arrays.

3. Final Answer:

   { $out: "targetCollection" } -> Option A

4. Quick Check:

   $out needs collection name string [OK]

Hint: Use simple string for collection name in $out [OK]

Common Mistakes:

• Using object or array instead of string for collection name
• Trying to specify database inside $out stage
• Adding extra braces or brackets around collection name

3. Given the aggregation pipeline below, what will be the result in the summary collection after running it?

db.sales.aggregate([
  { $group: { _id: "$category", total: { $sum: "$amount" } } },
  { $merge: { into: "summary", on: "_id", whenMatched: "replace", whenNotMatched: "insert" } }
])

medium

A. The summary collection will remain unchanged.

B. The summary collection will be completely replaced with the aggregation results.

C. The aggregation will fail due to missing $out stage.

D. The summary collection will contain documents with total sales per category, replacing existing matching documents.

Solution

1. Step 1: Understand the $group stage

   This groups sales by category and sums amounts, producing documents like { _id: category, total: sum }.

2. Step 2: Understand the $merge stage

   $merge writes results into summary collection, replacing documents with matching _id or inserting new ones.

3. Final Answer:

   summary collection updated with total sales per category, replacing matches -> Option D

4. Quick Check:

   $merge updates or inserts by key [OK]

Hint: Remember $merge updates matching keys, inserts new [OK]

Common Mistakes:

• Thinking $merge replaces entire collection like $out
• Assuming aggregation fails without $out
• Believing collection stays unchanged after $merge

4. You run this aggregation pipeline but get an error:

db.orders.aggregate([
  { $match: { status: "shipped" } },
  { $out: { collection: "shippedOrders" } }
])

What is the likely cause of the error?

medium

A. You cannot use $out after $match stage.

B. The $out stage syntax is incorrect; it should be a string, not an object.

C. The $match stage is invalid and causes the error.

D. The target collection shippedOrders does not exist.

Solution

1. Step 1: Check $out syntax

   $out requires the target collection name as a string, not an object with a collection field.

2. Step 2: Analyze error cause

   Using { collection: "shippedOrders" } causes syntax error; correct usage is { $out: "shippedOrders" }.

3. Final Answer:

   Incorrect $out syntax using object instead of string -> Option B

4. Quick Check:

   $out needs string collection name [OK]

Hint: Use string for $out target, not object [OK]

Common Mistakes:

• Assuming $match causes error
• Thinking $out requires existing collection
• Believing $out cannot follow $match

5. You want to aggregate user activity data and save results into a collection named userStats. You want to update existing documents by userId and insert new ones if missing, but keep other fields unchanged when matched. Which $merge option should you use?

hard

A. { into: "userStats", on: "userId", whenMatched: "merge", whenNotMatched: "insert" }

B. { into: "userStats", on: "userId", whenMatched: "replace", whenNotMatched: "insert" }

C. { into: "userStats", on: "userId", whenMatched: "fail", whenNotMatched: "insert" }

D. { into: "userStats", on: "userId", whenMatched: "keepExisting", whenNotMatched: "discard" }

Solution

1. Step 1: Understand whenMatched options

   whenMatched: "merge" updates only specified fields, keeping others unchanged; replace overwrites entire document.

2. Step 2: Choose correct $merge options

   To update existing documents by userId and insert new ones, use whenMatched: "merge" and whenNotMatched: "insert".

3. Final Answer:

   { into: "userStats", on: "userId", whenMatched: "merge", whenNotMatched: "insert" } -> Option A

4. Quick Check:

   whenMatched: "merge" updates fields, inserts new [OK]

Hint: Use whenMatched: "merge" to update fields, not replace [OK]

Common Mistakes:

• Using replace which overwrites entire documents
• Choosing fail which errors on matches
• Using invalid whenNotMatched options