Collection functions (length, flatten, merge) in Terraform - Deep Dive

Overview - Collection functions (length, flatten, merge)

What is it?

Collection functions in Terraform are tools that help you work with groups of values like lists and maps. The length function tells you how many items are in a collection. Flatten takes a list of lists and turns it into a single list with all the items. Merge combines multiple maps into one map. These functions make it easier to organize and use data in your infrastructure code.

Why it matters

Without these functions, managing groups of resources or settings would be slow and error-prone. You would have to count items manually, combine data by hand, or write complex code to handle nested lists. These functions save time and reduce mistakes, making your infrastructure more reliable and easier to maintain.

Where it fits

Before learning collection functions, you should understand basic Terraform syntax and how lists and maps work. After mastering these functions, you can explore more advanced data manipulation, like using for expressions and conditional logic to create dynamic infrastructure.

Mental Model

Core Idea

Collection functions let you count, combine, and simplify groups of values to manage infrastructure data easily.

Think of it like...

Imagine you have several boxes of LEGO bricks. Length tells you how many bricks are in a box, flatten is like pouring all bricks from multiple boxes into one big pile, and merge is like combining different sets of instructions into one guide.

┌─────────────┐      ┌─────────────┐      ┌─────────────┐
│ List of     │      │ List of     │      │ Map 1       │
│ lists       │      │ lists       │      │ {a=1, b=2}  │
│ [[1,2],[3]] │ ──▶  │ Flattened   │      ├─────────────┤
└─────────────┘      │ list [1,2,3]│      │ Map 2       │
                     └─────────────┘      │ {b=3, c=4}  │
                                          └─────────────┘
                                               │
                                               ▼
                                          ┌─────────────┐
                                          │ Merged map  │
                                          │ {a=1,b=3,c=4}│
                                          └─────────────┘

Build-Up - 6 Steps

1

Foundation

Understanding Terraform Collections

Concept: Learn what lists and maps are in Terraform and how they store multiple values.

In Terraform, a list is an ordered group of values like ["apple", "banana", "cherry"]. A map is a set of key-value pairs like {name = "server", size = "large"}. Collections help organize related data together.

Result

You can store multiple related values in one variable or resource attribute.

Understanding collections is essential because all collection functions operate on these data types.

2

Foundation

Basic Use of length Function

Concept: Learn how to count the number of items in a list or map using length.

The length function returns the number of elements in a list or keys in a map. For example, length([1,2,3]) returns 3. length({a=1,b=2}) returns 2.

Result

You can quickly find out how many items are in your collection.

Knowing the size of collections helps in loops, conditions, and validations in your infrastructure code.

3

Intermediate

Using flatten to Simplify Nested Lists

🤔Before reading on: do you think flatten changes the order of items or just removes nesting? Commit to your answer.

Concept: Flatten takes a list of lists and combines all inner lists into one single list without changing the order of items.

If you have a nested list like [["a", "b"], ["c"]], flatten returns ["a", "b", "c"]. This is useful when you get nested outputs from modules or resources and want a simple list.

Result

You get a single-level list with all elements in the original order.

Understanding flatten helps you handle complex data structures and simplifies data processing in Terraform.

4

Intermediate

Merging Maps with merge Function

🤔Before reading on: if two maps have the same key, do you think merge keeps the first or last value? Commit to your answer.

Concept: Merge combines multiple maps into one. If keys overlap, the value from the last map wins.

For example, merge({a=1,b=2}, {b=3,c=4}) results in {a=1,b=3,c=4}. This is useful to combine settings from different sources.

Result

You get one map with all keys and values, with later maps overriding earlier ones.

Knowing how merge handles conflicts prevents unexpected overwrites in your configurations.

5

Advanced

Combining Functions for Dynamic Configurations

🤔Before reading on: can you predict the result of length(flatten([[1],[2,3]]))? Commit to your answer.

Concept: You can use length, flatten, and merge together to create flexible and dynamic infrastructure definitions.

Example: length(flatten([[1],[2,3]])) returns 3 because flatten makes [1,2,3] and length counts 3. You can merge maps from different modules and count combined resources.

Result

You can write concise code that adapts to changing inputs and scales your infrastructure automatically.

Mastering function combinations unlocks powerful Terraform patterns for real-world infrastructure.

6

Expert

Performance and Limitations of Collection Functions

🤔Before reading on: do you think flatten or merge can handle very large collections without performance impact? Commit to your answer.

Concept: Collection functions are efficient but can slow down with very large or deeply nested data. Understanding their limits helps optimize Terraform runs.

Terraform evaluates these functions during plan and apply phases. Large nested lists or many merges can increase computation time. Also, merge only works on maps, flatten only on lists of lists.

Result

You learn to design your data structures to avoid performance bottlenecks and errors.

Knowing function limits helps you write scalable and maintainable Terraform code.

Under the Hood

Terraform parses your configuration and evaluates collection functions during the plan phase. Length counts elements by iterating over the collection. Flatten traverses nested lists to produce a single list by removing one level of nesting. Merge iterates over each map's keys and values, combining them into a new map, with later maps overriding earlier keys. These operations happen in memory within Terraform's evaluation engine before any infrastructure changes.

Why designed this way?

Terraform's collection functions were designed to simplify common data manipulation tasks without requiring external scripts. They follow predictable rules to keep configurations declarative and readable. Alternatives like manual loops or external tools would complicate workflows and reduce portability.

Terraform Configuration
        │
        ▼
  ┌───────────────┐
  │ Collection    │
  │ Functions     │
  └───────────────┘
    │      │       │
    │      │       │
    ▼      ▼       ▼
 ┌─────┐ ┌─────┐ ┌─────┐
 │length│ │flatten│ │merge│
 └─────┘ └─────┘ └─────┘
    │      │       │
    ▼      ▼       ▼
  Count  Single  Combined
  items  list    map
    │      │       │
    └──────┴───────┘
           │
           ▼
   Terraform Plan & Apply

Myth Busters - 4 Common Misconceptions

Quick: Does merge keep the first map's value or the last map's value when keys overlap? Commit to your answer.

Common Belief: Merge keeps the first map's value when keys overlap.

Reality: Merge uses the value from the last map for overlapping keys, overriding earlier ones.

Why it matters: Assuming the first value stays can cause unexpected configuration overrides and resource misconfigurations.

Quick: Does flatten change the order of elements in the list? Commit to your answer.

Common Belief: Flatten changes the order of elements when combining nested lists.

Reality: Flatten preserves the original order of elements while removing nesting.

Why it matters: Misunderstanding order can lead to bugs when order matters, such as in resource dependencies.

Quick: Can length be used on nested lists to count all elements inside? Commit to your answer.

Common Belief: Length counts all elements inside nested lists recursively.

Reality: Length only counts the top-level elements, not nested ones inside inner lists.

Why it matters: Expecting recursive counting can cause wrong assumptions about collection sizes and lead to logic errors.

Quick: Can merge combine lists like it does maps? Commit to your answer.

Common Belief: Merge can combine lists as it does maps.

Reality: Merge only works on maps; lists must be combined with other functions like concat or flatten.

Why it matters: Trying to merge lists with merge causes errors and breaks Terraform runs.

Expert Zone

1

Merge applies keys from later maps last, so order of arguments controls which values prevail in conflicts.

2

Flatten only removes one level of nesting; deeply nested lists require multiple flatten calls or other techniques.

3

Length on maps counts keys, not values, which can be confusing when maps contain nested collections.

When NOT to use

Avoid using flatten on very deeply nested or irregular lists; instead, restructure data or use for expressions. Do not use merge for lists; use concat or flatten instead. For very large collections, consider external data processing to improve performance.

Production Patterns

In production, merge is often used to combine default and environment-specific settings. Flatten helps when aggregating outputs from multiple modules. Length is used for validation and conditional resource creation based on collection size.

Connections

Functional Programming

Collection functions in Terraform are similar to functional programming concepts like map, reduce, and flatten.

Understanding functional programming helps grasp how Terraform processes collections declaratively and immutably.

Database Joins

Merge resembles a join operation in databases where two sets of key-value pairs combine based on keys.

Knowing database joins clarifies how merge handles overlapping keys and data combination.

Inventory Management

Counting items with length is like inventory counting in stores, and flattening nested lists is like unpacking boxes into shelves.

Relating to inventory helps understand why counting and flattening collections are practical and necessary.

Common Pitfalls

#1 Using merge on lists instead of maps.

Wrong approach:

merge(["a", "b"], ["c"])

Correct approach:

concat(["a", "b"], ["c"])

Root cause:

Confusing merge as a general collection combiner instead of map-specific.

#2 Expecting length to count all nested elements.

Wrong approach:

length([[1,2],[3,4]]) # expecting 4

Correct approach:

length(flatten([[1,2],[3,4]])) # returns 4

Root cause:

Not realizing length counts only top-level elements.

#3 Assuming merge keeps first map's values on key conflicts.

Wrong approach:

merge({a=1,b=2}, {b=3,c=4}) # expecting b=2

Correct approach:

merge({a=1,b=2}, {b=3,c=4}) # actual b=3

Root cause:

Misunderstanding merge's override order.

Key Takeaways

Terraform collection functions length, flatten, and merge help manage lists and maps efficiently.

Length counts top-level elements, flatten removes one level of list nesting, and merge combines maps with later values overriding earlier ones.

Using these functions correctly simplifies infrastructure code and reduces errors.

Misunderstanding their behavior can cause bugs, so knowing their exact operation is crucial.

Combining these functions enables powerful, dynamic Terraform configurations for real-world infrastructure.