Recursive CTE for graph traversal in PostgreSQL - Step-by-Step Execution

Concept Flow - Recursive CTE for graph traversal

Start with base nodes

↓

Recursive step: find neighbors

↓

Add new nodes to result

↓

Check if new nodes found

↓

Repeat recursion

Start with initial nodes, then repeatedly find connected neighbors until no new nodes are found.

Execution Sample

PostgreSQL

WITH RECURSIVE traverse AS (
  SELECT source, target, 1 AS depth
  FROM edges
  WHERE source = 1
  UNION
  SELECT e.source, e.target, t.depth + 1
  FROM edges e
  JOIN traverse t ON e.source = t.target
  WHERE t.depth < 3
)
SELECT * FROM traverse;

This query finds nodes reachable from node 1 up to 3 steps away using a recursive CTE.

Execution Table

Step	Action	New Rows Added	Current Result Set (source->target, depth)	Termination Check
1	Initialize base case: select edges where source=1	Edges (1->2), (1->3)	(1->2, 1), (1->3, 1)	New rows found, continue
2	Recursive step: find edges where source matches previous target	Edges (2->4), (3->5)	(1->2, 1), (1->3, 1), (2->4, 2), (3->5, 2)	New rows found, continue
3	Recursive step: find edges where source matches previous target	Edges (4->6), (5->6)	(1->2, 1), (1->3, 1), (2->4, 2), (3->5, 2), (4->6, 3), (5->6, 3)	Depth limit reached (3), stop recursion
4	No further recursion	No new rows	(1->2, 1), (1->3, 1), (2->4, 2), (3->5, 2), (4->6, 3), (5->6, 3)	No new rows, recursion ends

💡 Recursion stops because depth limit 3 is reached and no new rows are found.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
depth	N/A	1	2	3	3
result_set_count	0	2	4	6	6
new_rows_count	0	2	2	2	0

Key Moments - 3 Insights

Why does the recursion stop even though some nodes might still connect further?

Because the query limits recursion depth to 3 (see execution_table row 3), so it stops after reaching that depth.

How does the recursive part avoid infinite loops in cyclic graphs?

The recursion stops when no new rows are added (execution_table row 4), preventing infinite loops by not revisiting already found nodes.

Why do we join edges on e.source = t.target in the recursive step?

Because we want to find edges starting from nodes found in the previous step (t.target), extending the path forward.

Visual Quiz - 3 Questions

Look at the execution_table at Step 2, how many new rows are added?

A 2

B 3

C 4

D 1

💡 Hint

Check the 'New Rows Added' column for Step 2 in the execution_table.

At which step does the recursion stop due to reaching the depth limit?

A Step 1

B Step 3

C Step 4

D Step 2

💡 Hint

Look at the 'Termination Check' column describing depth limit in execution_table.

If the depth limit was removed, what would happen to the 'new_rows_count' after Step 3?

A It would decrease to zero immediately

B It would stay zero

C It would increase if more edges exist

D It would cause an error

💡 Hint

Refer to variable_tracker for 'new_rows_count' and understand recursion behavior without depth limit.

Concept Snapshot

WITH RECURSIVE cte_name AS (
  -- Base case: select starting nodes
  SELECT ...
  UNION
  -- Recursive step: join cte with edges to find neighbors
  SELECT ... FROM edges JOIN cte_name ON ...
  WHERE depth < max_depth
)
SELECT * FROM cte_name;

Use recursive CTEs to traverse graphs step-by-step, stopping when no new nodes or depth limit reached.

Full Transcript

Recursive Common Table Expressions (CTEs) in PostgreSQL allow you to traverse graphs by starting from base nodes and repeatedly finding connected neighbors. The process begins by selecting initial edges from a starting node. Then, the recursive part joins the current results with the edges table to find new connected nodes, increasing the depth each time. This repeats until no new nodes are found or a depth limit is reached, preventing infinite loops. The execution table shows each step adding new rows representing edges found at increasing depths. Variables like depth and counts of new rows track progress. Key points include understanding why recursion stops (depth limit or no new rows) and how the join condition extends the traversal. This method is powerful for exploring hierarchical or network data in SQL.

Practice

1. What is the main purpose of using a WITH RECURSIVE clause in PostgreSQL when working with graph data?

easy

A. To optimize query performance by indexing graph nodes

B. To create temporary tables for storing graph data

C. To repeatedly execute a query that references itself to explore connected nodes

D. To join two unrelated tables without conditions

Solution

1. Step 1: Understand the purpose of recursive CTEs

   Recursive CTEs allow a query to call itself repeatedly, which is useful for exploring connected data like graph nodes.

2. Step 2: Identify the correct use case

   Using WITH RECURSIVE lets you traverse graph structures step-by-step by referencing the CTE inside itself.

3. Final Answer:

   To repeatedly execute a query that references itself to explore connected nodes -> Option C

4. Quick Check:

   Recursive CTE = self-referencing query [OK]

Hint: Recursive CTEs repeat queries to explore connected data [OK]

Common Mistakes:

• Confusing recursive CTE with temporary tables
• Thinking it automatically optimizes performance
• Using it for unrelated table joins

2. Which of the following is the correct basic syntax to start a recursive CTE named path in PostgreSQL?

easy

A. WITH path RECURSIVE AS (SELECT ... UNION ALL SELECT ... FROM path ...)

B. CREATE RECURSIVE TABLE path AS SELECT ...

C. SELECT RECURSIVE * FROM path

D. WITH RECURSIVE path AS (SELECT ... UNION ALL SELECT ... FROM path ...)

Solution

1. Step 1: Recall the syntax for recursive CTEs

   The correct syntax starts with WITH RECURSIVE cte_name AS ( ... ), where the CTE references itself inside.

2. Step 2: Match the syntax options

   WITH RECURSIVE path AS (SELECT ... UNION ALL SELECT ... FROM path ...) correctly uses WITH RECURSIVE path AS (SELECT ... UNION ALL SELECT ... FROM path ...). Other options misuse keywords or order.

3. Final Answer:

   WITH RECURSIVE path AS (SELECT ... UNION ALL SELECT ... FROM path ...) -> Option D

4. Quick Check:

   WITH RECURSIVE + CTE name + self-reference [OK]

Hint: Start with WITH RECURSIVE cte_name AS (...) [OK]

Common Mistakes:

• Swapping RECURSIVE and CTE name order
• Using CREATE instead of WITH
• Omitting UNION ALL between parts

3. Given the table edges(source INT, target INT) with data:
(1, 2), (2, 3), (3, 4), (2, 5)
What will the following query return?

WITH RECURSIVE path AS (SELECT source, target FROM edges WHERE source = 1 UNION ALL SELECT e.source, e.target FROM edges e JOIN path p ON e.source = p.target) SELECT * FROM path ORDER BY source, target;

medium

A. Rows: (1,2), (2,3), (2,5), (3,4)

B. Rows: (1,2), (2,3), (3,4)

C. Rows: (1,2), (2,5)

D. Empty result

Solution

1. Step 1: Identify the anchor and recursive parts

   The anchor selects edges where source=1: (1,2). The recursive part joins edges where source matches previous target.

2. Step 2: Trace recursive expansion

   From (1,2), next edges with source=2 are (2,3) and (2,5). From (2,3), next edge with source=3 is (3,4). No further edges from 5 or 4.

3. Final Answer:

   Rows: (1,2), (2,3), (2,5), (3,4) -> Option A

4. Quick Check:

   Recursive join expands all reachable edges [OK]

Hint: Follow edges from start node stepwise [OK]

Common Mistakes:

• Ignoring multiple edges from same source
• Stopping recursion too early
• Misordering results

4. Consider this recursive CTE:

WITH RECURSIVE path(node) AS (SELECT 1 UNION ALL SELECT node + 1 FROM path WHERE node < 3) SELECT * FROM path;

What is wrong if this query runs forever without stopping?

medium

A. Not selecting from the CTE in the final SELECT

B. Missing a proper termination condition in the recursive part

C. Using UNION instead of UNION ALL

D. Incorrect anchor query syntax

Solution

1. Step 1: Analyze the recursive condition

   The recursive part adds 1 to node while node < 3, but due to UNION ALL accumulating duplicates of rows satisfying the condition, recursion continues indefinitely.

2. Step 2: Identify why it loops infinitely

   If the termination condition is incorrect or not reached, recursion never stops. Here, the condition is present but ineffective against duplicates, causing infinite loop.

3. Final Answer:

   Missing a proper termination condition in the recursive part -> Option B

4. Quick Check:

   Infinite recursion = no stop condition [OK]

Hint: Check recursive WHERE clause stops recursion [OK]

Common Mistakes:

• Confusing UNION and UNION ALL effects
• Ignoring anchor query correctness
• Forgetting to select from CTE

5. You have a table employees(id INT, manager_id INT) representing a company hierarchy. How would you write a recursive CTE to find all employees under manager with id = 10, including indirect reports?

hard

A. WITH RECURSIVE subordinates AS (
SELECT id FROM employees WHERE manager_id = 10
UNION ALL
SELECT e.id FROM employees e JOIN subordinates s ON e.manager_id = s.id
) SELECT * FROM subordinates;

B. WITH RECURSIVE subordinates AS (
SELECT id FROM employees WHERE id = 10
UNION ALL
SELECT e.id FROM employees e JOIN subordinates s ON e.id = s.manager_id
) SELECT * FROM subordinates;

C. WITH RECURSIVE subordinates AS (
SELECT id FROM employees WHERE manager_id = 10
UNION ALL
SELECT e.manager_id FROM employees e JOIN subordinates s ON e.id = s.id
) SELECT * FROM subordinates;

D. WITH RECURSIVE subordinates AS (
SELECT id FROM employees WHERE manager_id = 10
UNION ALL
SELECT e.id FROM employees e JOIN subordinates s ON e.id = s.manager_id
) SELECT * FROM subordinates;

Solution

1. Step 1: Identify the anchor query

   The anchor selects employees directly reporting to manager 10: WHERE manager_id = 10.

2. Step 2: Understand the recursive join

   The recursive part joins employees whose manager_id matches any previously found employee id, thus finding indirect reports.

3. Final Answer:

   WITH RECURSIVE subordinates AS ( SELECT id FROM employees WHERE manager_id = 10 UNION ALL SELECT e.id FROM employees e JOIN subordinates s ON e.manager_id = s.id ) SELECT * FROM subordinates; -> Option A

4. Quick Check:

   Join on e.manager_id = s.id to find subordinates [OK]

Hint: Join on manager_id = previous employee id to find reports [OK]

Common Mistakes:

• Swapping join conditions (id vs manager_id)
• Starting anchor with wrong WHERE clause
• Selecting wrong columns in recursive part