Provisioner failure behavior in Terraform - Commands & Configuration

Introduction

Sometimes when Terraform creates resources, it runs extra scripts called provisioners. These scripts can fail, and Terraform needs to know what to do next. This concept explains how Terraform handles failures in these provisioners and how you can control that behavior.

When you want to run a setup script on a virtual machine after it is created.

When you need to install software automatically on a new server during deployment.

When you want to copy files to a resource right after it is created.

When you want to control if Terraform should stop or continue when a provisioner script fails.

When you want to debug why a resource creation failed due to a script error.

Config File - main.tf

main.tf

resource "aws_instance" "example" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t2.micro"

  provisioner "local-exec" {
    command = "echo Hello from provisioner"
  }

  provisioner "remote-exec" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y nginx"
    ]

    on_failure = "continue"
  }
}

This Terraform file creates an AWS EC2 instance.

The local-exec provisioner runs a simple local command after the instance is created.

The remote-exec provisioner runs commands on the instance to update packages and install nginx.

The on_failure = "continue" setting tells Terraform to keep going even if the remote commands fail.

Commands

This command prepares Terraform to work with the AWS provider and downloads necessary plugins.

Terminal

terraform init

Expected Output

Initializing the backend...



Initializing provider plugins...

- Finding latest version of hashicorp/aws...

- Installing hashicorp/aws v4.0.0...

- Installed hashicorp/aws v4.0.0 (signed by HashiCorp)



Terraform has been successfully initialized!

This command creates the AWS instance and runs the provisioners. The apply runs without asking for confirmation.

Terminal

terraform apply -auto-approve

Expected Output

aws_instance.example: Creating...

aws_instance.example: Provisioning with 'local-exec'...

aws_instance.example (local-exec): Executing: ["sh" "-c" "echo Hello from provisioner"]

aws_instance.example: Provisioning with 'remote-exec'...

aws_instance.example (remote-exec): Connecting to remote host via SSH...

aws_instance.example (remote-exec): Running inline script

aws_instance.example (remote-exec): sudo: unable to resolve host ip-172-31-0-1

aws_instance.example (remote-exec): Reading package lists...

aws_instance.example (remote-exec): E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)

aws_instance.example (remote-exec): E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

aws_instance.example: Creation complete after 1m10s [id=i-0abcd1234efgh5678]



Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

→

-auto-approve - Skip manual approval to apply changes immediately

This command removes the AWS instance and cleans up resources without asking for confirmation.

Terminal

terraform destroy -auto-approve

Expected Output

aws_instance.example: Refreshing state... [id=i-0abcd1234efgh5678]

aws_instance.example: Destroying...

aws_instance.example: Still destroying... [id=i-0abcd1234efgh5678, 10s elapsed]

aws_instance.example: Destruction complete after 20s



Destroy complete! Resources: 1 destroyed.

→

-auto-approve - Skip manual approval to destroy resources immediately

Key Concept

If a provisioner fails, Terraform can either stop the whole process or continue based on the on_failure setting.

Common Mistakes

Not setting on_failure and expecting Terraform to continue after a provisioner error.

By default, Terraform stops if a provisioner fails, so the deployment halts unexpectedly.

Set on_failure = "continue" in the provisioner block if you want Terraform to keep going despite errors.

Using provisioners for tasks that can be done with native Terraform resources or cloud-init.

Provisioners are less reliable and harder to debug than native methods.

Use provisioners only as a last resort; prefer native Terraform features or cloud-init scripts.

Summary

Terraform provisioners run scripts on resources after creation to perform setup tasks.

If a provisioner fails, Terraform stops by default unless on_failure is set to continue.

Use terraform apply to create resources and run provisioners, and terraform destroy to clean up.

Practice

1.

What does the on_failure argument in a Terraform provisioner control?

easy

A. It controls what happens if the provisioner script or command fails.

B. It sets the timeout duration for the provisioner execution.

C. It defines the order in which provisioners run.

D. It specifies the operating system for the provisioner.

Solution

1. Step 1: Understand provisioner purpose

   Provisioners run scripts or commands after resource creation.

2. Step 2: Role of on_failure

   The on_failure argument controls what Terraform does if the provisioner fails.

3. Final Answer:

   It controls what happens if the provisioner script or command fails. -> Option A

4. Quick Check:

   on_failure controls failure behavior [OK]

Hint: Remember: on_failure decides action on provisioner failure [OK]

Common Mistakes:

• Confusing on_failure with timeout settings
• Thinking on_failure sets provisioner order
• Assuming on_failure defines OS type

2.

Which of the following is the correct syntax to make a provisioner continue even if it fails?

provisioner "local-exec" {
  command = "echo Hello"
  on_failure = ?
}

easy

A. "continue"

B. "fail"

C. "ignore"

D. "skip"

Solution

1. Step 1: Recall valid on_failure values

   Terraform supports fail and continue as valid on_failure values.

2. Step 2: Identify correct value to continue on failure

   To continue even if the provisioner fails, use continue.

3. Final Answer:

   "continue" -> Option A

4. Quick Check:

   on_failure = "continue" continues on failure [OK]

Hint: Use "continue" to ignore failure and proceed [OK]

Common Mistakes:

• Using invalid values like "ignore" or "skip"
• Confusing "fail" with continuing on failure
• Omitting quotes around the value

3.

Given this Terraform snippet, what happens if the local-exec provisioner command fails?

resource "null_resource" "example" {
  provisioner "local-exec" {
    command = "exit 1"
    on_failure = "fail"
  }
}

medium

A. Terraform retries the provisioner command automatically.

B. Terraform ignores the failure and continues applying other resources.

C. Terraform stops and reports an error, halting the apply process.

D. Terraform logs a warning but marks the resource as created.

Solution

1. Step 1: Analyze provisioner command and on_failure

   The command "exit 1" will fail. The on_failure is set to "fail".

2. Step 2: Understand effect of on_failure = "fail"

   This setting causes Terraform to stop and report an error if the provisioner fails.

3. Final Answer:

   Terraform stops and reports an error, halting the apply process. -> Option C

4. Quick Check:

   on_failure = "fail" stops on failure [OK]

Hint: Fail stops Terraform; continue lets it proceed [OK]

Common Mistakes:

• Assuming Terraform retries automatically
• Thinking failure is ignored with "fail"
• Believing resource is created despite failure

4.

Identify the error in this provisioner block that causes Terraform to ignore failures unexpectedly:

provisioner "remote-exec" {
  inline = ["false_command"]
  on_failure = fail
}

medium

A. The on_failure argument is not allowed in provisioners.

B. The on_failure value should be a string with quotes.

C. The provisioner type remote-exec is not supported.

D. The inline command is invalid and causes syntax error.

Solution

1. Step 1: Check on_failure syntax

   The value for on_failure must be a quoted string, e.g., "fail" or "continue".

2. Step 2: Identify impact of missing quotes

   Without quotes, Terraform treats it as an identifier, causing unexpected behavior or ignoring the setting.

3. Final Answer:

   The on_failure value should be a string with quotes. -> Option B

4. Quick Check:

   on_failure values require quotes [OK]

Hint: Always quote on_failure values "fail" or "continue" [OK]

Common Mistakes:

• Omitting quotes around on_failure values
• Confusing command errors with syntax errors
• Thinking on_failure is unsupported

5.

You want to run a local-exec provisioner that might fail but should not stop Terraform from creating other resources. Which configuration achieves this?

resource "null_resource" "example" {
  provisioner "local-exec" {
    command = "some_command"
    on_failure = ?
  }
}

hard

A. "fail" to stop on failure

B. "retry" to try again on failure

C. "ignore" to silently skip failure

D. "continue" to ignore failure and proceed

Solution

1. Step 1: Understand requirement to not stop Terraform

   The provisioner may fail but Terraform should continue creating other resources.

2. Step 2: Choose correct on_failure value

   Using on_failure = "continue" tells Terraform to ignore provisioner failure and proceed.

3. Final Answer:

   "continue" to ignore failure and proceed -> Option D

4. Quick Check:

   on_failure = "continue" allows continuation on failure [OK]

Hint: Use "continue" to keep going despite provisioner failure [OK]

Common Mistakes:

• Using unsupported values like "retry" or "ignore"
• Choosing "fail" which stops Terraform
• Not quoting the on_failure value