Graph lookup for recursive data in MongoDB - Step-by-Step Execution

Concept Flow - Graph lookup for recursive data

Start with root node

↓

Lookup connected nodes

↓

Add connected nodes to results

↓

For each connected node

↓

Repeat lookup recursively

↓

Stop when no new nodes found

↓

Return full recursive graph

Start from a root node, find connected nodes recursively, and collect all related nodes until no new nodes are found.

Execution Sample

MongoDB

db.nodes.aggregate([
  {
    $graphLookup: {
      from: "nodes",
      startWith: "$_id",
      connectFromField: "parent",
      connectToField: "_id",
      as: "ancestors"
    }
  }
])

This query recursively finds all ancestor nodes starting from the '_id' field of each document.

Execution Table

Step	Action	Current Node _id	Connected Nodes Found	Accumulated Results
1	Start with root node	5	parent: 3	[5]
2	Lookup connected nodes	3	parent: 1	[5, 3]
3	Lookup connected nodes	1	parent: null	[5, 3, 1]
4	No more connected nodes	null	none	[5, 3, 1]
5	Return full recursive graph	null	none	[5, 3, 1]

💡 No more connected nodes found, recursion ends.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
Current Node _id	null	5	3	1	null
Connected Nodes Found	[]	[3]	[1]	[]	[]
Accumulated Results	[]	[5]	[5, 3]	[5, 3, 1]	[5, 3, 1]

Key Moments - 2 Insights

Why does the recursion stop when the connected node is null?

Because in the execution_table at Step 3 and 4, no new connected nodes are found (parent is null), so the recursion ends.

How does $graphLookup avoid infinite loops in recursive data?

It tracks accumulated results (see Accumulated Results column) and stops when no new nodes are found, preventing revisiting the same nodes.

Visual Quiz - 3 Questions

Look at the execution_table, what is the Accumulated Results after Step 2?

A [3]

B [5, 3]

C [5]

D [5, 3, 1]

💡 Hint

Check the 'Accumulated Results' column in the execution_table row for Step 2.

At which step does the recursion find no more connected nodes?

A Step 2

B Step 3

C Step 4

D Step 1

💡 Hint

Look at the 'Connected Nodes Found' column in the execution_table to find where it becomes 'none'.

If the root node had no parent, what would be the Accumulated Results after Step 1?

A [root node id]

B [parent id]

C []

D null

💡 Hint

Refer to variable_tracker for how Accumulated Results starts with the root node id.

Concept Snapshot

Use $graphLookup to recursively find connected documents.
Start with a field (startWith).
Connect from one field to another (connectFromField, connectToField).
Collect all connected nodes in an array (as).
Stops when no new nodes are found to avoid infinite loops.

Full Transcript

This visual execution shows how MongoDB's $graphLookup works to find recursive data. Starting from a root node, it looks up connected nodes by matching fields. Each connected node is added to the results. The process repeats recursively for each connected node until no new nodes are found. The accumulated results grow step by step, and recursion stops when no more connections exist. This prevents infinite loops and collects the full recursive graph data.

Practice

1. What is the primary purpose of the $graphLookup stage in MongoDB aggregation?

easy

A. To create indexes on fields

B. To perform a simple filter on documents

C. To update multiple documents at once

D. To recursively search connected documents within the same collection

Solution

1. Step 1: Understand the role of $graphLookup

   $graphLookup is designed to recursively find related documents by following references within the same collection, useful for hierarchical data.

2. Step 2: Compare with other options

   Options B, C, and D describe other MongoDB operations unrelated to recursive lookup.

3. Final Answer:

   To recursively search connected documents within the same collection -> Option D

4. Quick Check:

   $graphLookup = recursive search [OK]

Hint: Remember $graphLookup is for recursive connected data search [OK]

Common Mistakes:

• Confusing $graphLookup with $match filtering
• Thinking $graphLookup updates documents
• Assuming $graphLookup creates indexes

2. Which of the following is the correct syntax to start a $graphLookup stage in a MongoDB aggregation pipeline?

easy

A. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "reportingHierarchy"}}

B. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}}

C. {"$graphLookup": {"from": "employees", "startWith": "managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "reportingHierarchy"}}

D. {"$graphLookup": {"from": "employees", "startWith": "$id", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}}

Solution

1. Step 1: Identify correct field references

   In $graphLookup, startWith uses the field to start recursion, usually a reference like "$managerId". The connectToField is the field in the foreign documents to match against the start value, here "_id". The connectFromField is the field in the foreign document used for the next recursion, here "managerId".

2. Step 2: Match syntax with options

   {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} correctly uses startWith: "$managerId", connectFromField: "managerId", and connectToField: "_id". Others swap these or use incorrect field names.

3. Final Answer:

   {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} -> Option B

4. Quick Check:

   Correct field mapping = {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} [OK]

Hint: Match startWith with $field, connectToField with foreign _id [OK]

Common Mistakes:

• Swapping connectFromField and connectToField
• Missing $ in startWith field
• Using wrong field names for connections

3. Given the following collection categories with documents:

{"_id": 1, "name": "Electronics", "parent": null}

{"_id": 2, "name": "Computers", "parent": 1}

{"_id": 3, "name": "Laptops", "parent": 2}

What will be the result of this aggregation?

[ { "$graphLookup": { "from": "categories", "startWith": "$parent", "connectFromField": "parent", "connectToField": "_id", "as": "ancestors" } } ]

for the document with _id: 3?

medium

A. An array 'ancestors' containing documents with _id 2 and 1

B. An empty array 'ancestors'

C. An array 'ancestors' containing only document with _id 1

D. A syntax error due to incorrect $graphLookup usage

Solution

1. Step 1: Understand the recursive lookup

   Starting from the document with _id 3, startWith uses its parent field which is 2. Then it finds documents where _id matches 2, then recursively finds parents of 2, which is 1.

2. Step 2: Trace the ancestors found

   The recursion finds documents with _id 2 and then _id 1, so the 'ancestors' array contains both documents.

3. Final Answer:

   An array 'ancestors' containing documents with _id 2 and 1 -> Option A

4. Quick Check:

   Recursive parents = [2, 1] [OK]

Hint: Trace startWith field value and follow connectToField recursively [OK]

Common Mistakes:

• Assuming only direct parent is returned
• Confusing connectFromField and connectToField
• Expecting syntax error incorrectly

4. Consider this aggregation pipeline snippet:

{ "$graphLookup": { "from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "team" } }

Why might this pipeline fail or return empty results when run on an employee document with no managerId field?

medium

A. Because as field name is invalid

B. Because connectFromField and connectToField are swapped

C. Because startWith is null or missing, recursion cannot start

D. Because from collection does not exist

Solution

1. Step 1: Check the startWith field value

   If the document has no managerId field, startWith evaluates to null or undefined, so the recursion has no starting point.

2. Step 2: Understand impact on results

   Without a valid start value, $graphLookup returns an empty array for 'team' because it cannot find connected documents.

3. Final Answer:

   Because startWith is null or missing, recursion cannot start -> Option C

4. Quick Check:

   Missing startWith value = empty results [OK]

Hint: Ensure startWith field exists and is not null [OK]

Common Mistakes:

• Thinking connectFromField/connectToField swap causes empty results
• Assuming 'as' field name causes failure
• Ignoring missing 'from' collection errors

5. You have a collection employees where each document has _id and managerId. You want to find all employees under a manager recursively, including the manager. Which $graphLookup configuration correctly achieves this?

hard

A. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}}

B. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}}

C. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}}

D. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}}

Solution

1. Step 1: Define the recursion direction

   To find all employees under a manager, start with the manager's _id. Then find employees whose managerId matches this _id, recursively.

2. Step 2: Match fields in $graphLookup

   startWith should be $_id (the manager's id). connectToField is managerId (field in employees to match), and connectFromField is _id (field to match against for next level).

3. Step 3: Verify options

   {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} matches this logic exactly. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}} swaps connectFromField and connectToField incorrectly. The options starting with $managerId are wrong for this use case.

4. Final Answer:

   {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} -> Option A

5. Quick Check:

   Start with manager _id, connectFromField = _id, connectToField = managerId = {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} [OK]

Hint: Start with manager _id, connectFromField = _id, connectToField = managerId [OK]

Common Mistakes:

• Swapping connectFromField and connectToField
• Starting with $managerId instead of $_id
• Confusing direction of recursion