Module composition patterns in Terraform - Deep Dive

Overview - Module composition patterns

What is it?

Module composition patterns in Terraform are ways to organize and connect reusable pieces of infrastructure code called modules. These patterns help you build complex infrastructure by combining smaller, manageable parts. Each module can represent a specific resource or a group of resources, and composition means putting these modules together to form a complete system. This approach makes infrastructure easier to understand, maintain, and reuse.

Why it matters

Without module composition patterns, infrastructure code becomes large, tangled, and hard to manage. Changes in one place can cause unexpected problems elsewhere. By using composition patterns, teams can work on smaller parts independently, reuse tested modules, and reduce errors. This leads to faster development, safer updates, and clearer infrastructure design that anyone can understand.

Where it fits

Before learning module composition patterns, you should understand basic Terraform concepts like resources, variables, outputs, and simple modules. After mastering composition patterns, you can explore advanced topics like module versioning, remote module sources, and Terraform workspaces for environment management.

Mental Model

Core Idea

Module composition patterns are like building with LEGO blocks, where small, reusable pieces snap together to create complex structures.

Think of it like...

Imagine building a house using LEGO bricks. Each brick is simple and has a clear shape, but when you connect many bricks in the right way, you get a complete house. Similarly, Terraform modules are like bricks, and composition patterns show you how to snap them together to build your infrastructure.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ Module A     │─────▶│ Module B     │─────▶│ Module C     │
│ (Network)   │      │ (Servers)   │      │ (Database)  │
└───────────────┘      └───────────────┘      └───────────────┘
         │                    │                    │
         ▼                    ▼                    ▼
    ┌───────────────────────────────────────────────┐
    │                Root Module                     │
    │  (Composes A, B, and C into full system)      │
    └───────────────────────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Terraform Modules Basics

Concept: Learn what a Terraform module is and how it groups resources.

A Terraform module is a folder with Terraform files that define resources. It can be as simple as one resource or many. Modules help organize code by grouping related resources together. You can call a module from another Terraform configuration to reuse its code.

Result

You can create and use simple modules to organize your infrastructure code.

Understanding modules as reusable building blocks is the foundation for composing larger infrastructure systems.

2

Foundation

Using Module Inputs and Outputs

Concept: Modules communicate with the outside using inputs (variables) and outputs.

Modules accept inputs through variables to customize their behavior. They expose outputs to share information with other modules or the root configuration. For example, a network module might output the subnet ID for other modules to use.

Result

You can customize modules and connect them by passing inputs and reading outputs.

Knowing how modules exchange data is key to linking them together in composition.

3

Intermediate

Root Module as Composition Center

🤔Before reading on: Do you think the root module should contain all resources directly or just compose other modules? Commit to your answer.

Concept: The root module acts as the main place that calls and connects other modules.

Instead of putting all resources in one place, the root module calls smaller modules and passes data between them. This keeps the root simple and lets each module focus on one part of the infrastructure. For example, the root module calls a network module, then a compute module that uses the network's outputs.

Result

Your infrastructure is split into clear parts, making it easier to manage and update.

Using the root module as a composition center helps separate concerns and improves clarity.

4

Intermediate

Module Composition Patterns: Nested Modules

🤔Before reading on: Do you think modules can call other modules inside them, or only the root module can? Commit to your answer.

Concept: Modules can call other modules inside themselves, creating nested structures.

A module can include calls to other modules, forming a tree of modules. This lets you build complex systems by layering modules. For example, a 'web app' module might call a 'network' module and a 'database' module internally, hiding complexity from the root module.

Result

You can build reusable higher-level modules that group smaller modules together.

Nested modules enable abstraction and reuse, making large infrastructure easier to handle.

5

Intermediate

Module Composition Patterns: Wrapper Modules

🤔Before reading on: Do you think wrapper modules add new resources or just organize existing modules? Commit to your answer.

Concept: Wrapper modules group multiple modules and can add extra resources or logic around them.

A wrapper module calls several modules and may add resources that connect or configure them. For example, a wrapper module might call a network module and a compute module, then add firewall rules that depend on both. This pattern helps enforce policies or add common features.

Result

You can create modules that enforce standards and combine modules with extra logic.

Wrapper modules help maintain consistency and add shared features across composed modules.

6

Advanced

Handling Dependencies Between Modules

🤔Before reading on: Do you think Terraform automatically knows the order to create modules, or do you need to specify dependencies? Commit to your answer.

Concept: Terraform uses outputs and inputs to understand dependencies and create resources in the right order.

When one module needs data from another, you pass outputs as inputs. Terraform builds a dependency graph and creates resources in order. Sometimes you need explicit dependencies using 'depends_on' if outputs are not enough. Managing dependencies ensures resources are ready before others use them.

Result

Your infrastructure deploys reliably without errors from missing resources.

Understanding and managing dependencies prevents deployment failures and race conditions.

7

Expert

Avoiding Common Pitfalls in Module Composition

🤔Before reading on: Do you think reusing modules without versioning is safe in production? Commit to your answer.

Concept: Proper versioning, input validation, and clear interfaces are critical for safe module composition.

In production, modules should be versioned to avoid unexpected changes. Inputs should be validated to catch errors early. Outputs should be stable and documented. Avoid circular dependencies and keep modules focused. Using remote module sources with version tags helps manage updates safely.

Result

Your composed infrastructure is stable, maintainable, and less prone to errors during updates.

Mastering these practices protects your infrastructure from subtle bugs and downtime.

Under the Hood

Terraform builds a dependency graph from module inputs and outputs, then plans and applies changes in order. Each module is a separate namespace with its own variables and resources. When you compose modules, Terraform merges their graphs, ensuring resources are created respecting dependencies. Internally, modules are folders with Terraform files, and calls to modules are like function calls passing arguments and receiving results.

Why designed this way?

Terraform modules were designed to promote code reuse and separation of concerns. Early infrastructure code was monolithic and hard to maintain. Modules allow teams to share tested components. Composition patterns evolved to manage complexity and enable collaboration. The design balances flexibility with simplicity, avoiding tight coupling between modules.

Root Module
  │
  ├─ Module A (Network)
  │     ├─ Resource 1
  │     └─ Resource 2
  ├─ Module B (Compute)
  │     ├─ Resource 3
  │     └─ Resource 4
  └─ Module C (Database)
        ├─ Resource 5
        └─ Resource 6

Terraform builds dependency graph:
Module A outputs → Module B inputs
Module B outputs → Module C inputs

Apply order: Module A → Module B → Module C

Myth Busters - 4 Common Misconceptions

Quick: Do you think modules must be nested only one level deep? Commit to yes or no.

Common Belief: Modules should only be called from the root module, not from other modules.

Reality: Modules can call other modules, allowing nested composition and better abstraction.

Why it matters: Limiting modules to one level reduces reuse and forces complex root modules, making maintenance harder.

Quick: Do you think passing outputs as inputs is optional for module dependencies? Commit to yes or no.

Common Belief: Terraform automatically knows dependencies between modules without passing outputs as inputs.

Reality: Terraform relies on explicit input-output connections to build the dependency graph correctly.

Why it matters: Without explicit connections, Terraform may create resources in the wrong order, causing failures.

Quick: Do you think using the latest module code without versioning is safe in production? Commit to yes or no.

Common Belief: Always using the latest module code ensures you get the newest features and fixes automatically.

Reality: Not versioning modules can introduce breaking changes unexpectedly, causing downtime or errors.

Why it matters: Versioning modules protects production environments from unplanned disruptions.

Quick: Do you think modules can share state directly with each other? Commit to yes or no.

Common Belief: Modules can directly share state or variables without passing through inputs and outputs.

Reality: Modules are isolated; they communicate only via inputs and outputs, not shared state.

Why it matters: Assuming shared state leads to hidden dependencies and unpredictable behavior.

Expert Zone

1

Modules should have clear, minimal interfaces to avoid tight coupling and ease updates.

2

Using wrapper modules to enforce organizational policies helps maintain compliance across teams.

3

Explicitly managing module versions and using semantic versioning prevents accidental breaking changes.

When NOT to use

Avoid deep nested modules when simple flat structures suffice, as excessive nesting can complicate debugging. For very dynamic or conditional infrastructure, consider using Terraform workspaces or separate state files instead of complex module composition.

Production Patterns

In production, teams use a root module per environment that composes stable, versioned modules from a shared registry. Wrapper modules enforce security policies and naming conventions. Modules are tested independently and updated via CI/CD pipelines to ensure safe rollouts.

Connections

Software Design Patterns

Module composition in Terraform builds on the same principles of modularity and separation of concerns found in software design.

Understanding software design patterns helps grasp why breaking infrastructure into modules improves maintainability and reuse.

Microservices Architecture

Both module composition and microservices break complex systems into smaller, independent parts that communicate via defined interfaces.

Knowing microservices concepts clarifies why clear inputs and outputs between modules reduce coupling and improve scalability.

Manufacturing Assembly Lines

Module composition is like assembling products on a line where each station adds a part, ensuring order and quality.

Seeing module composition as an assembly line highlights the importance of dependencies and order in building infrastructure.

Common Pitfalls

#1 Passing outputs directly without validating inputs causes errors.

Wrong approach:

module "compute" {
source = "./modules/compute"
subnet_id = module.network.subnet_id
instance_type = ""
}

Correct approach:

module "compute" {
source = "./modules/compute"
subnet_id = module.network.subnet_id
instance_type = var.instance_type
}

Root cause:

Not validating or passing required inputs leads to empty or incorrect values causing deployment failures.

#2 Not versioning modules leads to unexpected changes.

Wrong approach:

module "network" {
source = "git::https://example.com/network.git"
}

Correct approach:

module "network" {
source = "git::https://example.com/network.git?ref=v1.2.0"
}

Root cause:

Using unversioned module sources means any code change affects your infrastructure without control.

#3 Creating circular dependencies between modules causes errors.

Wrong approach:

module "A" {
source = "./A"
input_from_B = module.B.output
}

module "B" {
source = "./B"
input_from_A = module.A.output
}

Correct approach:

Refactor modules to remove circular references by redesigning inputs or merging modules.

Root cause:

Modules depending on each other's outputs create loops Terraform cannot resolve.

Key Takeaways

Terraform modules are reusable building blocks that organize infrastructure code.

Composition patterns connect modules through inputs and outputs to build complex systems.

The root module acts as the main composer, calling and linking smaller modules.

Proper dependency management and versioning are essential for reliable infrastructure.

Advanced patterns like nested and wrapper modules enable abstraction and policy enforcement.