FOREACH for array iteration in PostgreSQL - Deep Dive

Overview - FOREACH for array iteration

What is it?

FOREACH is a control structure in PostgreSQL used to loop through each element of an array. It allows you to perform actions on each item inside the array one by one. This helps when you want to process or analyze all elements without writing repetitive code. It is especially useful in PL/pgSQL functions and procedures.

Why it matters

Without FOREACH, handling arrays in PostgreSQL would require manual indexing or complex loops, which can be error-prone and harder to read. FOREACH simplifies array processing, making code cleaner and easier to maintain. This improves productivity and reduces bugs when working with collections of data inside the database.

Where it fits

Before learning FOREACH, you should understand basic SQL queries, arrays in PostgreSQL, and simple PL/pgSQL control structures like loops and conditionals. After mastering FOREACH, you can explore more advanced array functions, set-returning functions, and performance optimization techniques for array processing.

Mental Model

Core Idea

FOREACH lets you visit each item in an array one at a time to perform actions, just like reading a list from start to end.

Think of it like...

Imagine you have a basket of apples and you want to check each apple for freshness. Instead of grabbing apples randomly, you pick one apple at a time from the basket and inspect it carefully before moving to the next. FOREACH works the same way with array elements.

┌─────────────┐
│   ARRAY     │
│ [a, b, c]   │
└─────┬───────┘
      │
      ▼
┌─────────────┐
│ FOREACH LOOP│
│ 1st: a      │
│ 2nd: b      │
│ 3rd: c      │
└─────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding PostgreSQL Arrays

Concept: Learn what arrays are in PostgreSQL and how they store multiple values in a single column.

In PostgreSQL, an array is a data type that can hold multiple values of the same type, like integers or text, inside one column. For example, you can have an integer array: {1,2,3} or a text array: {'apple','banana','cherry'}. Arrays let you group related data together easily.

Result

You can store and retrieve multiple values in one column using arrays.

Knowing arrays is essential because FOREACH operates directly on these collections to process each element.

2

Foundation

Basics of PL/pgSQL Loops

Concept: Learn how to write simple loops in PostgreSQL's procedural language to repeat actions.

PL/pgSQL supports loops like LOOP, WHILE, and FOR to repeat commands. For example, a FOR loop can count from 1 to 5 and perform an action each time. Loops help automate repetitive tasks inside functions.

Result

You can write code that repeats actions multiple times inside the database.

Understanding loops prepares you to use FOREACH, which is a specialized loop for arrays.

3

Intermediate

Using FOREACH to Iterate Arrays

🤔Before reading on: do you think FOREACH requires manual index handling or automatically processes each element? Commit to your answer.

Concept: FOREACH automatically loops through each element of an array without needing manual index tracking.

FOREACH syntax looks like this: FOREACH element_variable IN ARRAY array_variable LOOP -- actions using element_variable END LOOP; This runs the loop body once for each element in the array, assigning the current element to element_variable.

Result

Each array element is processed in order without extra code for indexing.

Knowing FOREACH handles indexing internally reduces errors and simplifies array processing code.

4

Intermediate

FOREACH with Different Data Types

🤔Before reading on: do you think FOREACH works only with text arrays or with any array type? Commit to your answer.

Concept: FOREACH works with arrays of any data type supported by PostgreSQL, such as integers, text, or custom types.

You can use FOREACH with integer arrays, text arrays, or even arrays of composite types. For example: DECLARE nums integer[] := ARRAY[10, 20, 30]; num integer; BEGIN FOREACH num IN ARRAY nums LOOP RAISE NOTICE 'Number: %', num; END LOOP; END; This prints each number in the array.

Result

FOREACH processes each element regardless of its type.

Understanding type flexibility makes FOREACH a powerful tool for many array processing tasks.

5

Intermediate

FOREACH with Slice and Reverse Options

🤔Before reading on: do you think FOREACH can iterate arrays backwards or only forwards? Commit to your answer.

Concept: FOREACH supports options to iterate over a slice of the array or in reverse order.

You can specify a slice of the array to loop over using the SLICE option, or iterate backwards using REVERSE. For example: FOREACH elem IN ARRAY arr SLICE 2..4 LOOP -- process elements 2 to 4 END LOOP; FOREACH elem IN ARRAY arr REVERSE LOOP -- process elements from last to first END LOOP;

Result

You can control which parts of the array to process and the direction of iteration.

Knowing these options lets you write more precise and flexible array loops.

6

Advanced

FOREACH in Real-World Functions

🤔Before reading on: do you think FOREACH is only for simple tasks or can it handle complex logic inside loops? Commit to your answer.

Concept: FOREACH can be used inside complex PL/pgSQL functions to process arrays with conditional logic and database operations.

In production, you might use FOREACH to update multiple rows, validate data, or build dynamic queries. For example: FOREACH item IN ARRAY input_array LOOP IF item IS NOT NULL THEN PERFORM some_function(item); END IF; END LOOP; This shows how FOREACH integrates with other logic.

Result

FOREACH enables powerful, readable array processing inside database functions.

Understanding how to combine FOREACH with conditions and queries unlocks practical database programming.

7

Expert

Performance and Internals of FOREACH

🤔Before reading on: do you think FOREACH creates a copy of the array or works directly on it? Commit to your answer.

Concept: FOREACH operates directly on the array without copying it, and its performance depends on array size and loop body complexity.

Internally, FOREACH uses PostgreSQL's array access functions to fetch elements one by one. It does not duplicate the array, which saves memory. However, large arrays or complex loop bodies can impact performance. Using FOREACH with REVERSE or SLICE does not add significant overhead.

Result

FOREACH is efficient for array iteration but should be used thoughtfully with large data.

Knowing FOREACH's internal behavior helps optimize database functions and avoid performance pitfalls.

Under the Hood

FOREACH uses PostgreSQL's internal array subscripting functions to access each element sequentially. It sets the loop variable to the current element and executes the loop body. The array is not copied; instead, elements are fetched on demand. The loop can handle NULL elements and supports options like SLICE and REVERSE by adjusting the index range and direction internally.

Why designed this way?

FOREACH was designed to simplify array processing in PL/pgSQL by abstracting manual index management. This reduces programmer errors and improves code clarity. Alternatives like manual FOR loops with indexes were more error-prone and verbose. The design balances ease of use with performance by avoiding unnecessary array copying.

┌─────────────┐
│ Input Array │
│ [e1, e2, e3]│
└─────┬───────┘
      │
      ▼
┌─────────────┐
│ FOREACH     │
│ Loop Start  │
├─────────────┤
│ Index i=1   │
│ element=e1  │
│ Execute body│
│ Index i=2   │
│ element=e2  │
│ Execute body│
│ Index i=3   │
│ element=e3  │
│ Execute body│
└─────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does FOREACH require you to manually track the current index of the array? Commit to yes or no.

Common Belief: FOREACH requires manual handling of array indexes to access elements.

Reality: FOREACH automatically manages the current element internally and does not require manual index tracking.

Why it matters: Believing you must track indexes leads to more complex and error-prone code, defeating FOREACH's purpose.

Quick: Can FOREACH modify the original array elements directly? Commit to yes or no.

Common Belief: FOREACH can change the original array elements during iteration.

Reality: FOREACH only reads elements; it cannot modify the original array in place during iteration.

Why it matters: Expecting in-place modification can cause confusion and bugs; to change arrays, you must build a new array or update separately.

Quick: Does FOREACH create a copy of the array before looping? Commit to yes or no.

Common Belief: FOREACH duplicates the entire array internally before processing.

Reality: FOREACH accesses the original array elements directly without copying the array.

Why it matters: Thinking FOREACH copies arrays may cause unnecessary concern about memory usage or performance.

Quick: Is FOREACH limited to text arrays only? Commit to yes or no.

Common Belief: FOREACH only works with text arrays in PostgreSQL.

Reality: FOREACH works with arrays of any data type supported by PostgreSQL.

Why it matters: Limiting FOREACH to text arrays restricts its usefulness and leads to missed opportunities for code reuse.

Expert Zone

1

FOREACH does not support modifying the array elements directly; to change elements, you must build a new array or use other functions.

2

Using FOREACH with large arrays can impact performance; sometimes set-based SQL operations are more efficient.

3

FOREACH's SLICE and REVERSE options allow fine control over iteration, but combining them requires careful index understanding.

When NOT to use

FOREACH is not ideal when you need to perform set-based operations that can be done with SQL queries directly, as set operations are usually faster. Also, avoid FOREACH for very large arrays where performance is critical; consider unnesting arrays and using joins instead.

Production Patterns

In production, FOREACH is commonly used inside PL/pgSQL functions to validate array inputs, perform row-by-row processing, or call other functions per element. It is often combined with conditional logic and exception handling to build robust data processing routines.

Connections

Set-Based SQL Operations

FOREACH is a procedural alternative to set-based operations that work on entire tables or arrays at once.

Understanding FOREACH helps appreciate when to use procedural loops versus set-based queries for better performance and clarity.

Iterator Pattern (Software Design)

FOREACH embodies the iterator pattern by providing a way to access elements sequentially without exposing the underlying structure.

Recognizing FOREACH as an iterator helps understand its role in abstracting array traversal in database programming.

Assembly Line Processing (Manufacturing)

Like an assembly line processes items one by one, FOREACH processes array elements sequentially to apply operations.

Seeing FOREACH as an assembly line clarifies how each element is handled individually in order.

Common Pitfalls

#1 Trying to modify array elements directly inside FOREACH loop.

Wrong approach:

FOREACH elem IN ARRAY arr LOOP
elem := elem + 1; -- attempt to change element
END LOOP;

Correct approach:

DECLARE
new_arr integer[] := '{}';
elem integer;
BEGIN
FOREACH elem IN ARRAY arr LOOP
new_arr := array_append(new_arr, elem + 1);
END LOOP;
END;

Root cause:

Misunderstanding that FOREACH loop variable is a copy, not a reference to the original array element.

#2 Using FOREACH on a NULL array without checking.

Wrong approach:

FOREACH elem IN ARRAY null_array LOOP
RAISE NOTICE '%', elem;
END LOOP;

Correct approach:

IF null_array IS NOT NULL THEN
FOREACH elem IN ARRAY null_array LOOP
RAISE NOTICE '%', elem;
END LOOP;
END IF;

Root cause:

Not handling NULL arrays causes runtime errors because FOREACH cannot iterate over NULL.

#3 Manually managing indexes inside FOREACH loop.

Wrong approach:

DECLARE
i integer := 1;
BEGIN
FOREACH elem IN ARRAY arr LOOP
RAISE NOTICE 'Index %: %', i, elem;
i := i + 1;
END LOOP;
END;

Correct approach:

FOR i IN array_lower(arr,1)..array_upper(arr,1) LOOP
RAISE NOTICE 'Index %: %', i, arr[i];
END LOOP;

Root cause:

Confusing FOREACH with FOR loops that use indexes; FOREACH does not provide index tracking.

Key Takeaways

FOREACH is a simple and powerful way to loop through each element of an array in PostgreSQL without manual indexing.

It works with arrays of any data type and supports options to iterate slices or reverse order.

FOREACH loop variables are copies of elements; you cannot modify the original array elements directly inside the loop.

Using FOREACH improves code readability and reduces errors compared to manual array processing.

Understanding when to use FOREACH versus set-based operations is key for writing efficient database functions.