Graph lookup for recursive data in MongoDB - Mini Project: Build & Apply

Graph Lookup for Recursive Data in MongoDB

📖 Scenario: You are managing a company's employee database. Each employee may have a manager, and managers themselves can have their own managers, forming a hierarchy. You want to find all the subordinates under a specific manager, including indirect reports.

🎯 Goal: Build a MongoDB aggregation query using $graphLookup to recursively find all employees reporting to a given manager.

📋 What You'll Learn

Create a MongoDB collection named employees with documents containing _id, name, and manager_id fields.

Add a variable managerId to specify the starting manager's _id for the recursive search.

Write an aggregation pipeline using $graphLookup to find all employees reporting directly or indirectly to the manager with manager_id equal to managerId.

Include the subordinates field in the output documents containing the recursive results.

💡 Why This Matters

🌍 Real World

Companies often need to query hierarchical employee data to understand reporting structures and manage teams effectively.

💼 Career

Knowing how to use recursive queries like <code>$graphLookup</code> in MongoDB is valuable for backend developers and data engineers working with hierarchical or graph data.

1

Create the employees collection with sample data

Create a MongoDB collection called employees and insert these exact documents: { _id: 1, name: "Alice", manager_id: null }, { _id: 2, name: "Bob", manager_id: 1 }, { _id: 3, name: "Charlie", manager_id: 1 }, { _id: 4, name: "David", manager_id: 2 }, { _id: 5, name: "Eve", manager_id: 2 }.

MongoDB

// Insert the employees documents into the employees collection
// Your code here

Hint

Use db.employees.insertMany() with an array of documents.

2

Define the managerId variable for the lookup

Create a variable called managerId and set it to 1 to represent the manager Alice's _id.

MongoDB

db.employees.insertMany([
  { _id: 1, name: "Alice", manager_id: null },
  { _id: 2, name: "Bob", manager_id: 1 },
  { _id: 3, name: "Charlie", manager_id: 1 },
  { _id: 4, name: "David", manager_id: 2 },
  { _id: 5, name: "Eve", manager_id: 2 }
])

// Define the managerId variable
// Your code here

Hint

Use const managerId = 1 to set the variable.

3

Write the aggregation pipeline with $graphLookup

Write an aggregation pipeline on employees that uses $match to find the document with _id equal to managerId, then uses $graphLookup to recursively find all employees where manager_id matches _id in the hierarchy. Store the results in a field called subordinates.

MongoDB

db.employees.insertMany([
  { _id: 1, name: "Alice", manager_id: null },
  { _id: 2, name: "Bob", manager_id: 1 },
  { _id: 3, name: "Charlie", manager_id: 1 },
  { _id: 4, name: "David", manager_id: 2 },
  { _id: 5, name: "Eve", manager_id: 2 }
])

const managerId = 1

// Write the aggregation pipeline with $graphLookup
// Your code here

Hint

Use $graphLookup with from, startWith, connectFromField, connectToField, and as fields.

4

Run the aggregation and assign the result

Run the aggregation pipeline on employees using aggregate(pipeline) and assign the result to a variable called result.

MongoDB

db.employees.insertMany([
  { _id: 1, name: "Alice", manager_id: null },
  { _id: 2, name: "Bob", manager_id: 1 },
  { _id: 3, name: "Charlie", manager_id: 1 },
  { _id: 4, name: "David", manager_id: 2 },
  { _id: 5, name: "Eve", manager_id: 2 }
])

const managerId = 1

const pipeline = [
  { $match: { _id: managerId } },
  {
    $graphLookup: {
      from: "employees",
      startWith: "$_id",
      connectFromField: "_id",
      connectToField: "manager_id",
      as: "subordinates"
    }
  }
]

// Run the aggregation and assign to result
// Your code here

Hint

Use const result = db.employees.aggregate(pipeline) to run the query.

Practice

1. What is the primary purpose of the $graphLookup stage in MongoDB aggregation?

easy

A. To create indexes on fields

B. To perform a simple filter on documents

C. To update multiple documents at once

D. To recursively search connected documents within the same collection

Solution

1. Step 1: Understand the role of $graphLookup

   $graphLookup is designed to recursively find related documents by following references within the same collection, useful for hierarchical data.

2. Step 2: Compare with other options

   Options B, C, and D describe other MongoDB operations unrelated to recursive lookup.

3. Final Answer:

   To recursively search connected documents within the same collection -> Option D

4. Quick Check:

   $graphLookup = recursive search [OK]

Hint: Remember $graphLookup is for recursive connected data search [OK]

Common Mistakes:

• Confusing $graphLookup with $match filtering
• Thinking $graphLookup updates documents
• Assuming $graphLookup creates indexes

2. Which of the following is the correct syntax to start a $graphLookup stage in a MongoDB aggregation pipeline?

easy

A. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "reportingHierarchy"}}

B. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}}

C. {"$graphLookup": {"from": "employees", "startWith": "managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "reportingHierarchy"}}

D. {"$graphLookup": {"from": "employees", "startWith": "$id", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}}

Solution

1. Step 1: Identify correct field references

   In $graphLookup, startWith uses the field to start recursion, usually a reference like "$managerId". The connectToField is the field in the foreign documents to match against the start value, here "_id". The connectFromField is the field in the foreign document used for the next recursion, here "managerId".

2. Step 2: Match syntax with options

   {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} correctly uses startWith: "$managerId", connectFromField: "managerId", and connectToField: "_id". Others swap these or use incorrect field names.

3. Final Answer:

   {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} -> Option B

4. Quick Check:

   Correct field mapping = {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "reportingHierarchy"}} [OK]

Hint: Match startWith with $field, connectToField with foreign _id [OK]

Common Mistakes:

• Swapping connectFromField and connectToField
• Missing $ in startWith field
• Using wrong field names for connections

3. Given the following collection categories with documents:

{"_id": 1, "name": "Electronics", "parent": null}

{"_id": 2, "name": "Computers", "parent": 1}

{"_id": 3, "name": "Laptops", "parent": 2}

What will be the result of this aggregation?

[ { "$graphLookup": { "from": "categories", "startWith": "$parent", "connectFromField": "parent", "connectToField": "_id", "as": "ancestors" } } ]

for the document with _id: 3?

medium

A. An array 'ancestors' containing documents with _id 2 and 1

B. An empty array 'ancestors'

C. An array 'ancestors' containing only document with _id 1

D. A syntax error due to incorrect $graphLookup usage

Solution

1. Step 1: Understand the recursive lookup

   Starting from the document with _id 3, startWith uses its parent field which is 2. Then it finds documents where _id matches 2, then recursively finds parents of 2, which is 1.

2. Step 2: Trace the ancestors found

   The recursion finds documents with _id 2 and then _id 1, so the 'ancestors' array contains both documents.

3. Final Answer:

   An array 'ancestors' containing documents with _id 2 and 1 -> Option A

4. Quick Check:

   Recursive parents = [2, 1] [OK]

Hint: Trace startWith field value and follow connectToField recursively [OK]

Common Mistakes:

• Assuming only direct parent is returned
• Confusing connectFromField and connectToField
• Expecting syntax error incorrectly

4. Consider this aggregation pipeline snippet:

{ "$graphLookup": { "from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "team" } }

Why might this pipeline fail or return empty results when run on an employee document with no managerId field?

medium

A. Because as field name is invalid

B. Because connectFromField and connectToField are swapped

C. Because startWith is null or missing, recursion cannot start

D. Because from collection does not exist

Solution

1. Step 1: Check the startWith field value

   If the document has no managerId field, startWith evaluates to null or undefined, so the recursion has no starting point.

2. Step 2: Understand impact on results

   Without a valid start value, $graphLookup returns an empty array for 'team' because it cannot find connected documents.

3. Final Answer:

   Because startWith is null or missing, recursion cannot start -> Option C

4. Quick Check:

   Missing startWith value = empty results [OK]

Hint: Ensure startWith field exists and is not null [OK]

Common Mistakes:

• Thinking connectFromField/connectToField swap causes empty results
• Assuming 'as' field name causes failure
• Ignoring missing 'from' collection errors

5. You have a collection employees where each document has _id and managerId. You want to find all employees under a manager recursively, including the manager. Which $graphLookup configuration correctly achieves this?

hard

A. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}}

B. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}}

C. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}}

D. {"$graphLookup": {"from": "employees", "startWith": "$managerId", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}}

Solution

1. Step 1: Define the recursion direction

   To find all employees under a manager, start with the manager's _id. Then find employees whose managerId matches this _id, recursively.

2. Step 2: Match fields in $graphLookup

   startWith should be $_id (the manager's id). connectToField is managerId (field in employees to match), and connectFromField is _id (field to match against for next level).

3. Step 3: Verify options

   {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} matches this logic exactly. {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "managerId", "connectToField": "_id", "as": "team"}} swaps connectFromField and connectToField incorrectly. The options starting with $managerId are wrong for this use case.

4. Final Answer:

   {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} -> Option A

5. Quick Check:

   Start with manager _id, connectFromField = _id, connectToField = managerId = {"$graphLookup": {"from": "employees", "startWith": "$_id", "connectFromField": "_id", "connectToField": "managerId", "as": "team"}} [OK]

Hint: Start with manager _id, connectFromField = _id, connectToField = managerId [OK]

Common Mistakes:

• Swapping connectFromField and connectToField
• Starting with $managerId instead of $_id
• Confusing direction of recursion