Plan and apply separation in pipelines in Terraform - Deep Dive

Overview - Plan and apply separation in pipelines

What is it?

Plan and apply separation in pipelines means splitting the process of preparing changes and making those changes into separate steps. In Terraform, this usually means running 'terraform plan' first to see what will change, then running 'terraform apply' to actually make those changes. This separation helps avoid mistakes and gives a clear preview before changing real resources.

Why it matters

Without separating planning and applying, you risk making unexpected changes to your cloud resources, which can cause downtime or data loss. This separation acts like a safety check, letting you review changes before they happen. It also helps teams work together safely by reviewing plans before applying them.

Where it fits

Before learning this, you should understand basic Terraform commands and how infrastructure as code works. After this, you can learn about automating pipelines with CI/CD tools and managing Terraform state securely.

Mental Model

Core Idea

Separating planning and applying in pipelines lets you preview changes safely before making them, reducing errors and improving collaboration.

Think of it like...

It's like writing a draft of a letter first to check for mistakes before sending the final version to someone important.

┌───────────────┐      ┌───────────────┐
│ terraform plan│─────▶│ terraform apply│
│ (preview)     │      │ (execute)     │
└───────────────┘      └───────────────┘

Build-Up - 7 Steps

1

Foundation

Understand terraform plan basics

Concept: Learn what 'terraform plan' does and why it is important.

The 'terraform plan' command shows what changes Terraform will make to your infrastructure without actually making them. It compares your current state with your configuration files and lists additions, changes, or deletions.

Result

You get a detailed list of planned changes without affecting any real resources.

Understanding that 'terraform plan' previews changes helps you avoid surprises and mistakes before touching live infrastructure.

2

Foundation

Understand terraform apply basics

Concept: Learn what 'terraform apply' does and how it uses the plan.

'terraform apply' takes the planned changes and makes them happen in your cloud environment. It creates, updates, or deletes resources as defined in your configuration.

Result

Your infrastructure changes to match your Terraform configuration.

Knowing that 'terraform apply' executes changes helps you see why planning first is a safe practice.

3

Intermediate

Separate plan and apply in pipelines

🤔Before reading on: do you think running plan and apply together is safer or separating them is safer? Commit to your answer.

Concept: Learn why splitting planning and applying into different pipeline steps improves safety and control.

In a pipeline, run 'terraform plan' first and save its output as a plan file. Then, in a separate step, run 'terraform apply' using that saved plan file. This ensures the apply step only executes exactly what was planned.

Result

The pipeline prevents unexpected changes by applying only reviewed plans.

Separating these steps creates a checkpoint for review and approval, reducing risks in automated deployments.

4

Intermediate

Use saved plan files for consistency

🤔Before reading on: do you think running 'terraform apply' without a saved plan file can cause different changes than planned? Commit to yes or no.

Concept: Learn how saving the plan output to a file ensures the apply step matches the plan exactly.

Run 'terraform plan -out=planfile' to save the plan. Then run 'terraform apply planfile' to apply exactly that plan. This avoids drift between planning and applying.

Result

Apply uses the exact plan, preventing surprises from changes in configuration or environment between steps.

Using saved plan files guarantees the apply step is predictable and repeatable.

5

Intermediate

Integrate plan/apply separation in CI/CD

Concept: Learn how to build pipeline stages that separate planning and applying with manual approval.

Create pipeline stages: one runs 'terraform plan' and shows the output for review; another waits for approval before running 'terraform apply' with the saved plan file. This allows human checks before changes.

Result

Your pipeline safely automates infrastructure changes with controlled approvals.

Adding manual approval between plan and apply balances automation with safety and team collaboration.

6

Advanced

Handle state locking and concurrency

🤔Before reading on: do you think multiple apply steps running at once can cause problems? Commit to yes or no.

Concept: Learn how Terraform state locking prevents concurrent applies that could corrupt infrastructure state.

Terraform uses state locking in backends like S3 with DynamoDB or Terraform Cloud to prevent multiple applies at the same time. This ensures only one apply runs, avoiding conflicts.

Result

Your pipeline avoids race conditions and state corruption during apply.

Understanding state locking is key to safely running separated apply steps in parallel pipelines.

7

Expert

Manage drift and re-planning surprises

🤔Before reading on: do you think the plan you saved can become outdated before apply? Commit to yes or no.

Concept: Learn about drift—changes outside Terraform—and how it affects plan/apply separation.

If infrastructure changes outside Terraform after planning but before applying, the saved plan may no longer match reality. This can cause apply failures or unexpected results. Experts use monitoring and frequent re-planning to detect and handle drift.

Result

You avoid applying outdated plans that could cause errors or resource conflicts.

Knowing drift risks helps you design pipelines that re-validate plans or alert teams before applying.

Under the Hood

Terraform keeps a state file representing the current infrastructure. 'terraform plan' compares this state with the desired configuration and creates a plan of changes. When you save this plan to a file, it records the exact steps Terraform will take. 'terraform apply' reads this plan file and executes those steps, updating the state file accordingly. State locking mechanisms prevent multiple applies from running simultaneously to avoid conflicts.

Why designed this way?

Separating plan and apply was designed to reduce risk by allowing review before changes. Saving the plan ensures consistency between preview and execution. State locking was added to prevent race conditions and state corruption in collaborative environments. Alternatives like applying directly without planning were rejected because they increase chances of errors and unexpected changes.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Terraform     │       │ Saved Plan    │       │ Terraform     │
│ Configuration │──────▶│ File (plan)   │──────▶│ Apply Reads   │
└───────────────┘       └───────────────┘       │ Plan and     │
                                                  │ Updates State │
                                                  └───────────────┘

┌───────────────┐
│ State Locking │
│ Prevents      │
│ Concurrent    │
│ Applies       │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does running 'terraform apply' without a saved plan always apply exactly what you expect? Commit yes or no.

Common Belief: Running 'terraform apply' without a saved plan file is safe and always applies what you planned.

Reality: Without a saved plan, 'terraform apply' re-calculates changes at runtime, which can differ from what you saw in 'terraform plan'.

Why it matters: This can cause unexpected changes or resource deletions, leading to downtime or data loss.

Quick: Can multiple 'terraform apply' commands run safely at the same time? Commit yes or no.

Common Belief: You can run multiple 'terraform apply' commands in parallel without issues.

Reality: Running multiple applies concurrently can corrupt the Terraform state and cause conflicting changes.

Why it matters: State corruption can break your infrastructure management and require manual recovery.

Quick: If infrastructure changes outside Terraform after planning, will the saved plan still be valid? Commit yes or no.

Common Belief: Once you have a saved plan, it remains valid until you apply it, regardless of external changes.

Reality: External changes cause drift, making the saved plan outdated and possibly causing apply failures or errors.

Why it matters: Applying outdated plans can cause resource conflicts or failed deployments.

Quick: Is manual approval between plan and apply always necessary? Commit yes or no.

Common Belief: Manual approval is optional and slows down automation unnecessarily.

Reality: Manual approval adds a safety checkpoint that prevents accidental or harmful changes in production.

Why it matters: Skipping approval can lead to unreviewed changes causing outages or security issues.

Expert Zone

1

Terraform plan files are binary and tied to the exact Terraform version and provider versions; mismatches can cause apply failures.

2

State locking depends on backend support; some backends do not support locking, increasing risk in concurrent pipelines.

3

Drift detection requires monitoring external changes and may need periodic re-planning or alerts to maintain pipeline safety.

When NOT to use

Separating plan and apply is less useful in small, single-user projects where manual commands are run directly. In such cases, running 'terraform apply' with auto-approval may be faster. Also, in rapid prototyping, strict separation can slow iteration. Alternatives include using 'terraform apply -auto-approve' for quick changes or using Terraform Cloud's run tasks for integrated checks.

Production Patterns

In production, pipelines separate plan and apply with manual approval gates, use saved plan files for consistency, and enforce state locking with remote backends like S3+DynamoDB or Terraform Cloud. Teams integrate plan outputs into pull requests for code review and use drift detection tools to maintain state accuracy.

Connections

Git Pull Request Reviews

Builds-on

Separating plan and apply in pipelines is like code reviews in Git; both provide a chance to review changes before merging or applying, improving safety and collaboration.

Transaction Commit in Databases

Same pattern

Just like databases separate transaction preparation and commit to ensure data integrity, Terraform separates planning and applying to ensure infrastructure integrity.

Quality Control in Manufacturing

Builds-on

Planning and applying separation mirrors quality control steps where products are inspected before final assembly, preventing defects from reaching customers.

Common Pitfalls

#1 Applying changes without saving and using a plan file.

Wrong approach:

terraform plan
terraform apply

Correct approach:

terraform plan -out=planfile
terraform apply planfile

Root cause:

Not saving the plan file means apply recalculates changes, risking unexpected modifications.

#2 Running multiple apply commands simultaneously in a pipeline.

Wrong approach:

Parallel jobs both run 'terraform apply' at the same time.

Correct approach:

Use state locking and serialize apply steps to run one at a time.

Root cause:

Ignoring state locking causes race conditions and state corruption.

#3 Skipping manual approval between plan and apply in production pipelines.

Wrong approach:

Pipeline automatically runs 'terraform apply' right after 'terraform plan' without review.

Correct approach:

Add a manual approval step after plan before apply.

Root cause:

Assuming automation alone is safe without human checks increases risk of errors.

Key Takeaways

Separating 'terraform plan' and 'terraform apply' in pipelines lets you preview changes safely before making them.

Saving the plan output to a file ensures the apply step executes exactly what was reviewed.

State locking prevents concurrent applies that could corrupt infrastructure state.

Manual approval between plan and apply adds a critical safety checkpoint in production.

Understanding drift and re-planning helps avoid applying outdated or conflicting changes.

Practice

1. What is the main benefit of separating terraform plan and terraform apply in pipelines?

easy

A. It speeds up the deployment by combining steps

B. It skips the need for state files

C. It automatically fixes errors during apply

D. It allows you to review changes before applying them

Solution

1. Step 1: Understand the purpose of terraform plan

   This command shows what changes Terraform will make without applying them.

2. Step 2: Understand the purpose of terraform apply

   This command applies the changes to the infrastructure based on the plan.

3. Final Answer:

   It allows you to review changes before applying them -> Option D

4. Quick Check:

   Plan shows changes first = B [OK]

Hint: Plan shows changes, apply makes them live [OK]

Common Mistakes:

• Thinking plan applies changes
• Believing apply skips state files
• Assuming steps run faster combined

2. Which command correctly saves a Terraform plan to a file named myplan.tfplan?

easy

A. terraform apply -out=myplan.tfplan

B. terraform save myplan.tfplan

C. terraform plan -out=myplan.tfplan

D. terraform plan save myplan.tfplan

Solution

1. Step 1: Identify the correct syntax for saving a plan

   The correct command uses terraform plan -out=filename to save the plan.

2. Step 2: Check each option

   terraform plan -out=myplan.tfplan matches the correct syntax exactly.

3. Final Answer:

   terraform plan -out=myplan.tfplan -> Option C

4. Quick Check:

   Plan with -out saves file = C [OK]

Hint: Use 'plan -out=' to save plans, not apply [OK]

Common Mistakes:

• Using apply instead of plan to save
• Using incorrect command like 'save'
• Missing the '-out=' flag

3. Given these commands run in order:
terraform plan -out=planfile
terraform apply planfile
What happens when you run terraform apply planfile?

medium

A. Terraform applies exactly the changes saved in planfile

B. Terraform ignores planfile and creates a new plan

C. Terraform applies changes but may differ from planfile

D. Terraform throws an error because planfile is not a valid command

Solution

1. Step 1: Understand the role of the saved plan file

   The plan file contains the exact changes Terraform will apply.

2. Step 2: Understand how terraform apply planfile works

   This command applies the saved plan exactly, without recalculating changes.

3. Final Answer:

   Terraform applies exactly the changes saved in planfile -> Option A

4. Quick Check:

   Apply saved plan = exact changes [OK]

Hint: Apply with plan file applies exact saved changes [OK]

Common Mistakes:

• Thinking apply recalculates plan
• Assuming apply ignores plan file
• Believing apply with plan file causes errors

4. You run terraform apply myplan.tfplan but get an error saying the plan file is invalid. What is the most likely cause?

medium

A. You forgot to run terraform init before apply

B. The plan file was created with a different Terraform version

C. You used terraform plan without -out flag

D. The plan file is empty because no changes were detected

Solution

1. Step 1: Identify compatibility issues with plan files

   Plan files are version-specific and may be invalid if Terraform versions differ.

2. Step 2: Check other options

   Running init is required but usually causes different errors; missing -out means no plan file saved; empty plan files do not cause invalid file errors.

3. Final Answer:

   The plan file was created with a different Terraform version -> Option B

4. Quick Check:

   Version mismatch causes invalid plan file error [OK]

Hint: Check Terraform versions match when using saved plans [OK]

Common Mistakes:

• Assuming missing init causes invalid plan file error
• Thinking empty plan files cause invalid file errors
• Confusing missing -out with invalid plan file

5. You want to implement a pipeline that separates planning and applying Terraform changes. Which sequence of commands ensures you can review the plan before applying exactly those changes?

hard

A. terraform plan -out=planfile && terraform apply planfile

B. terraform apply && terraform plan -out=planfile

C. terraform plan && terraform apply

D. terraform apply -out=planfile && terraform apply planfile

Solution

1. Step 1: Identify correct order for plan and apply separation

   First, run terraform plan -out=planfile to save the plan and review it.

2. Step 2: Apply the saved plan exactly

   Then, run terraform apply planfile to apply the reviewed changes exactly.

3. Final Answer:

   terraform plan -out=planfile && terraform apply planfile -> Option A

4. Quick Check:

   Plan with -out then apply saved plan = A [OK]

Hint: Plan with -out then apply saved plan file [OK]

Common Mistakes:

• Applying before planning
• Not saving plan with -out
• Using apply -out which is invalid