Plan and apply separation in pipelines in Terraform - Deep Dive

Overview - Plan and apply separation in pipelines

What is it?

Plan and apply separation in pipelines means splitting the process of preparing changes and making those changes into separate steps. In Terraform, this usually means running 'terraform plan' first to see what will change, then running 'terraform apply' to actually make those changes. This separation helps avoid mistakes and gives a clear preview before changing real resources.

Why it matters

Without separating planning and applying, you risk making unexpected changes to your cloud resources, which can cause downtime or data loss. This separation acts like a safety check, letting you review changes before they happen. It also helps teams work together safely by reviewing plans before applying them.

Where it fits

Before learning this, you should understand basic Terraform commands and how infrastructure as code works. After this, you can learn about automating pipelines with CI/CD tools and managing Terraform state securely.

Mental Model

Core Idea

Separating planning and applying in pipelines lets you preview changes safely before making them, reducing errors and improving collaboration.

Think of it like...

It's like writing a draft of a letter first to check for mistakes before sending the final version to someone important.

┌───────────────┐      ┌───────────────┐
│ terraform plan│─────▶│ terraform apply│
│ (preview)     │      │ (execute)     │
└───────────────┘      └───────────────┘

Build-Up - 7 Steps

1

Foundation

Understand terraform plan basics

Concept: Learn what 'terraform plan' does and why it is important.

The 'terraform plan' command shows what changes Terraform will make to your infrastructure without actually making them. It compares your current state with your configuration files and lists additions, changes, or deletions.

Result

You get a detailed list of planned changes without affecting any real resources.

Understanding that 'terraform plan' previews changes helps you avoid surprises and mistakes before touching live infrastructure.

2

Foundation

Understand terraform apply basics

Concept: Learn what 'terraform apply' does and how it uses the plan.

'terraform apply' takes the planned changes and makes them happen in your cloud environment. It creates, updates, or deletes resources as defined in your configuration.

Result

Your infrastructure changes to match your Terraform configuration.

Knowing that 'terraform apply' executes changes helps you see why planning first is a safe practice.

3

Intermediate

Separate plan and apply in pipelines

🤔Before reading on: do you think running plan and apply together is safer or separating them is safer? Commit to your answer.

Concept: Learn why splitting planning and applying into different pipeline steps improves safety and control.

In a pipeline, run 'terraform plan' first and save its output as a plan file. Then, in a separate step, run 'terraform apply' using that saved plan file. This ensures the apply step only executes exactly what was planned.

Result

The pipeline prevents unexpected changes by applying only reviewed plans.

Separating these steps creates a checkpoint for review and approval, reducing risks in automated deployments.

4

Intermediate

Use saved plan files for consistency

🤔Before reading on: do you think running 'terraform apply' without a saved plan file can cause different changes than planned? Commit to yes or no.

Concept: Learn how saving the plan output to a file ensures the apply step matches the plan exactly.

Run 'terraform plan -out=planfile' to save the plan. Then run 'terraform apply planfile' to apply exactly that plan. This avoids drift between planning and applying.

Result

Apply uses the exact plan, preventing surprises from changes in configuration or environment between steps.

Using saved plan files guarantees the apply step is predictable and repeatable.

5

Intermediate

Integrate plan/apply separation in CI/CD

Concept: Learn how to build pipeline stages that separate planning and applying with manual approval.

Create pipeline stages: one runs 'terraform plan' and shows the output for review; another waits for approval before running 'terraform apply' with the saved plan file. This allows human checks before changes.

Result

Your pipeline safely automates infrastructure changes with controlled approvals.

Adding manual approval between plan and apply balances automation with safety and team collaboration.

6

Advanced

Handle state locking and concurrency

🤔Before reading on: do you think multiple apply steps running at once can cause problems? Commit to yes or no.

Concept: Learn how Terraform state locking prevents concurrent applies that could corrupt infrastructure state.

Terraform uses state locking in backends like S3 with DynamoDB or Terraform Cloud to prevent multiple applies at the same time. This ensures only one apply runs, avoiding conflicts.

Result

Your pipeline avoids race conditions and state corruption during apply.

Understanding state locking is key to safely running separated apply steps in parallel pipelines.

7

Expert

Manage drift and re-planning surprises

🤔Before reading on: do you think the plan you saved can become outdated before apply? Commit to yes or no.

Concept: Learn about drift—changes outside Terraform—and how it affects plan/apply separation.

If infrastructure changes outside Terraform after planning but before applying, the saved plan may no longer match reality. This can cause apply failures or unexpected results. Experts use monitoring and frequent re-planning to detect and handle drift.

Result

You avoid applying outdated plans that could cause errors or resource conflicts.

Knowing drift risks helps you design pipelines that re-validate plans or alert teams before applying.

Under the Hood

Terraform keeps a state file representing the current infrastructure. 'terraform plan' compares this state with the desired configuration and creates a plan of changes. When you save this plan to a file, it records the exact steps Terraform will take. 'terraform apply' reads this plan file and executes those steps, updating the state file accordingly. State locking mechanisms prevent multiple applies from running simultaneously to avoid conflicts.

Why designed this way?

Separating plan and apply was designed to reduce risk by allowing review before changes. Saving the plan ensures consistency between preview and execution. State locking was added to prevent race conditions and state corruption in collaborative environments. Alternatives like applying directly without planning were rejected because they increase chances of errors and unexpected changes.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Terraform     │       │ Saved Plan    │       │ Terraform     │
│ Configuration │──────▶│ File (plan)   │──────▶│ Apply Reads   │
└───────────────┘       └───────────────┘       │ Plan and     │
                                                  │ Updates State │
                                                  └───────────────┘

┌───────────────┐
│ State Locking │
│ Prevents      │
│ Concurrent    │
│ Applies       │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does running 'terraform apply' without a saved plan always apply exactly what you expect? Commit yes or no.

Common Belief: Running 'terraform apply' without a saved plan file is safe and always applies what you planned.

Reality: Without a saved plan, 'terraform apply' re-calculates changes at runtime, which can differ from what you saw in 'terraform plan'.

Why it matters: This can cause unexpected changes or resource deletions, leading to downtime or data loss.

Quick: Can multiple 'terraform apply' commands run safely at the same time? Commit yes or no.

Common Belief: You can run multiple 'terraform apply' commands in parallel without issues.

Reality: Running multiple applies concurrently can corrupt the Terraform state and cause conflicting changes.

Why it matters: State corruption can break your infrastructure management and require manual recovery.

Quick: If infrastructure changes outside Terraform after planning, will the saved plan still be valid? Commit yes or no.

Common Belief: Once you have a saved plan, it remains valid until you apply it, regardless of external changes.

Reality: External changes cause drift, making the saved plan outdated and possibly causing apply failures or errors.

Why it matters: Applying outdated plans can cause resource conflicts or failed deployments.

Quick: Is manual approval between plan and apply always necessary? Commit yes or no.

Common Belief: Manual approval is optional and slows down automation unnecessarily.

Reality: Manual approval adds a safety checkpoint that prevents accidental or harmful changes in production.

Why it matters: Skipping approval can lead to unreviewed changes causing outages or security issues.

Expert Zone

1

Terraform plan files are binary and tied to the exact Terraform version and provider versions; mismatches can cause apply failures.

2

State locking depends on backend support; some backends do not support locking, increasing risk in concurrent pipelines.

3

Drift detection requires monitoring external changes and may need periodic re-planning or alerts to maintain pipeline safety.

When NOT to use

Separating plan and apply is less useful in small, single-user projects where manual commands are run directly. In such cases, running 'terraform apply' with auto-approval may be faster. Also, in rapid prototyping, strict separation can slow iteration. Alternatives include using 'terraform apply -auto-approve' for quick changes or using Terraform Cloud's run tasks for integrated checks.

Production Patterns

In production, pipelines separate plan and apply with manual approval gates, use saved plan files for consistency, and enforce state locking with remote backends like S3+DynamoDB or Terraform Cloud. Teams integrate plan outputs into pull requests for code review and use drift detection tools to maintain state accuracy.

Connections

Git Pull Request Reviews

Builds-on

Separating plan and apply in pipelines is like code reviews in Git; both provide a chance to review changes before merging or applying, improving safety and collaboration.

Transaction Commit in Databases

Same pattern

Just like databases separate transaction preparation and commit to ensure data integrity, Terraform separates planning and applying to ensure infrastructure integrity.

Quality Control in Manufacturing

Builds-on

Planning and applying separation mirrors quality control steps where products are inspected before final assembly, preventing defects from reaching customers.

Common Pitfalls

#1 Applying changes without saving and using a plan file.

Wrong approach:

terraform plan
terraform apply

Correct approach:

terraform plan -out=planfile
terraform apply planfile

Root cause:

Not saving the plan file means apply recalculates changes, risking unexpected modifications.

#2 Running multiple apply commands simultaneously in a pipeline.

Wrong approach:

Parallel jobs both run 'terraform apply' at the same time.

Correct approach:

Use state locking and serialize apply steps to run one at a time.

Root cause:

Ignoring state locking causes race conditions and state corruption.

#3 Skipping manual approval between plan and apply in production pipelines.

Wrong approach:

Pipeline automatically runs 'terraform apply' right after 'terraform plan' without review.

Correct approach:

Add a manual approval step after plan before apply.

Root cause:

Assuming automation alone is safe without human checks increases risk of errors.

Key Takeaways

Separating 'terraform plan' and 'terraform apply' in pipelines lets you preview changes safely before making them.

Saving the plan output to a file ensures the apply step executes exactly what was reviewed.

State locking prevents concurrent applies that could corrupt infrastructure state.

Manual approval between plan and apply adds a critical safety checkpoint in production.

Understanding drift and re-planning helps avoid applying outdated or conflicting changes.