Azure Storage backend in Terraform - Deep Dive

Overview - Azure Storage backend

What is it?

Azure Storage backend is a way to store Terraform state files securely in Microsoft Azure's cloud storage service. Instead of keeping state files on your local computer, you save them in Azure Storage, which allows multiple users and systems to share and manage infrastructure state safely. This helps Terraform know what resources it has created and what changes to make next.

Why it matters

Without a shared and secure place to keep Terraform state, teams can overwrite each other's work or lose track of infrastructure changes. Azure Storage backend solves this by providing a reliable, centralized storage that supports locking and versioning. This prevents conflicts and data loss, making infrastructure management safer and more efficient.

Where it fits

Before learning Azure Storage backend, you should understand basic Terraform concepts like state files and providers. After mastering this, you can explore advanced Terraform workflows, remote state management with other backends, and automation pipelines that use Terraform with Azure.

Mental Model

Core Idea

Azure Storage backend is like a shared, secure filing cabinet in the cloud where Terraform keeps its record of infrastructure changes so everyone can work together without confusion.

Think of it like...

Imagine a group of friends sharing a single notebook to track their plans. Instead of each having their own copy, they write in the same notebook stored in a locked box that only they can open. This way, everyone sees the latest plan and no one accidentally erases or duplicates notes.

┌───────────────────────────────┐
│        Terraform Client        │
└──────────────┬────────────────┘
               │
               ▼
┌───────────────────────────────┐
│     Azure Storage Account      │
│ ┌───────────────┐             │
│ │ Blob Container│             │
│ │  (State Files)│             │
│ └───────────────┘             │
└───────────────────────────────┘

Build-Up - 6 Steps

1

Foundation

What is Terraform State File

Concept: Terraform state file is a record that keeps track of all the resources Terraform manages.

Terraform uses a file called 'terraform.tfstate' to remember what infrastructure it created and how it looks now. This file is essential because Terraform compares it with your configuration to know what to add, change, or remove.

Result

You understand that Terraform state is the memory of your infrastructure.

Knowing that Terraform state is the source of truth helps you realize why managing it carefully is critical.

2

Foundation

Local vs Remote State Storage

Concept: Terraform state can be stored locally or remotely, each with pros and cons.

By default, Terraform saves the state file on your computer. This works for single users but causes problems when multiple people work together. Remote state storage means saving the state file in a shared place like Azure Storage, so everyone accesses the same file.

Result

You see why local state is risky for teams and why remote state is safer.

Understanding the limits of local state storage motivates the need for remote backends.

3

Intermediate

Configuring Azure Storage Backend in Terraform

🤔Before reading on: do you think configuring backend requires creating resources first or can it be done before any resources exist? Commit to your answer.

Concept: You can configure Terraform to use Azure Storage as backend by specifying storage account details and container.

To use Azure Storage backend, you add a 'backend' block in your Terraform configuration specifying the storage account name, container name, and key (file name). Example: terraform { backend "azurerm" { resource_group_name = "myResourceGroup" storage_account_name = "mystorageaccount" container_name = "tfstate" key = "terraform.tfstate" } } This tells Terraform where to save and read the state file remotely.

Result

Terraform will store state files in Azure Storage instead of locally.

Knowing that backend configuration points Terraform to a shared state location is key to enabling collaboration.

4

Intermediate

State Locking and Consistency

🤔Before reading on: do you think Azure Storage backend automatically prevents two users from changing state at the same time? Commit to yes or no.

Concept: Azure Storage backend supports state locking to avoid conflicts when multiple users run Terraform simultaneously.

When Terraform uses Azure Storage backend, it creates a lock file in the storage container during operations. This lock prevents others from making changes until the first operation finishes. This avoids race conditions and corrupted state files.

Result

Terraform operations are safer and less likely to cause errors in team environments.

Understanding state locking explains how Terraform prevents conflicting changes in shared environments.

5

Advanced

Managing Access and Security

🤔Before reading on: do you think anyone with Azure Storage account access can modify Terraform state? Commit to your answer.

Concept: Proper access control and security settings are essential to protect Terraform state in Azure Storage.

You should use Azure role-based access control (RBAC) to limit who can read or write to the storage container. Enabling encryption at rest and in transit protects sensitive data. Using managed identities or service principals for authentication ensures secure automated access.

Result

Terraform state is protected from unauthorized access or accidental changes.

Knowing how to secure backend storage prevents leaks of sensitive infrastructure details.

6

Expert

Handling State Migration and Versioning

🤔Before reading on: do you think changing backend configuration automatically moves existing state files? Commit to yes or no.

Concept: Migrating existing local state to Azure Storage backend and managing state versions requires careful steps.

When switching to Azure Storage backend, you must run 'terraform init -migrate-state' to move local state to remote storage safely. Azure Storage supports blob versioning, which can help recover previous state versions if needed. Understanding these helps maintain state integrity during changes.

Result

You can safely migrate and manage Terraform state without losing data or causing conflicts.

Knowing migration and versioning techniques avoids common pitfalls during backend changes.

Under the Hood

Terraform backend configuration tells Terraform where to read and write the state file. When using Azure Storage backend, Terraform uses Azure's Blob Storage REST API to upload, download, and lock the state file. Locking is implemented by creating a lease on a blob, which prevents others from modifying it simultaneously. Terraform serializes state changes and ensures atomic updates to avoid corruption.

Why designed this way?

Azure Storage backend was designed to leverage Azure's reliable, scalable blob storage and built-in locking mechanisms to provide a robust remote state solution. Alternatives like local files or other backends lacked centralized control or locking, leading to conflicts. Using Azure Storage fits naturally for teams already using Azure cloud.

┌───────────────┐       ┌───────────────────────────┐
│ Terraform CLI │──────▶│ Azure Storage Blob Service │
└──────┬────────┘       └─────────────┬─────────────┘
       │                            │
       │ 1. Acquire lease (lock)    │
       │ 2. Read/Write state blob   │
       │ 3. Release lease           │
       ▼                            ▼
┌───────────────┐             ┌───────────────┐
│ Local Config  │             │ Blob Container│
└───────────────┘             └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does configuring Azure Storage backend automatically create the storage account and container? Commit to yes or no.

Common Belief: Terraform will create the Azure Storage account and container automatically when you configure the backend.

Reality: Terraform requires the Azure Storage account and container to exist before configuring the backend; it does not create them automatically.

Why it matters: Assuming automatic creation leads to errors during initialization and confusion about missing resources.

Quick: Can multiple users run Terraform apply at the same time safely with Azure Storage backend? Commit to yes or no.

Common Belief: Azure Storage backend allows multiple users to run Terraform apply simultaneously without issues.

Reality: Azure Storage backend uses locking to prevent concurrent operations; simultaneous applies will wait or fail to avoid state corruption.

Why it matters: Ignoring locking can cause race conditions and corrupted state files, breaking infrastructure management.

Quick: Is the Terraform state file encrypted by default in Azure Storage backend? Commit to yes or no.

Common Belief: Terraform state files stored in Azure Storage backend are encrypted by default and fully secure without extra setup.

Reality: While Azure Storage supports encryption at rest, you must enable it and configure access controls; Terraform itself does not encrypt state files.

Why it matters: Assuming default encryption can expose sensitive data if storage security is not properly configured.

Quick: Does changing the backend configuration in Terraform automatically migrate existing state files? Commit to yes or no.

Common Belief: Changing backend configuration automatically moves existing state files to the new backend.

Reality: You must explicitly run migration commands like 'terraform init -migrate-state' to move state files safely.

Why it matters: Failing to migrate state properly can cause Terraform to lose track of resources, leading to accidental resource recreation or deletion.

Expert Zone

1

Azure Storage backend locking uses blob leases which have a timeout; understanding lease renewal is important to avoid unexpected lock releases during long operations.

2

Using Azure AD authentication with managed identities for backend access improves security and avoids storing credentials in Terraform configs.

3

Blob versioning in Azure Storage can be leveraged to recover previous state versions, but requires enabling and managing lifecycle policies.

When NOT to use

Avoid Azure Storage backend if your infrastructure is not on Azure or if you require multi-cloud state management; consider Terraform Cloud or HashiCorp Consul for cross-cloud or advanced state workflows.

Production Patterns

In production, teams combine Azure Storage backend with Azure DevOps pipelines using service principals for automated runs, enforce RBAC policies on storage accounts, and use separate containers or keys per environment to isolate state files.

Connections

Git Version Control

Both manage shared state and changes collaboratively with locking or conflict prevention.

Understanding how Git prevents conflicting edits helps grasp why Terraform uses locking in remote backends.

Database Transactions

Terraform state locking is similar to database transactions that lock rows to prevent concurrent conflicting updates.

Knowing database locking concepts clarifies how Terraform ensures state consistency during concurrent operations.

Supply Chain Management

Both track the current state of complex systems and coordinate changes safely among multiple actors.

Seeing Terraform state as a supply chain ledger helps appreciate the importance of accurate, shared state for smooth operations.

Common Pitfalls

#1 Trying to configure Azure Storage backend without creating the storage account and container first.

Wrong approach:

terraform {
backend "azurerm" {
resource_group_name = "myGroup"
storage_account_name = "nonexistentaccount"
container_name = "tfstate"
key = "terraform.tfstate"
}
}

Correct approach:

Create the storage account and container in Azure first, then configure backend:

# Azure CLI commands
az storage account create --name mystorageaccount --resource-group myGroup --location eastus --sku Standard_LRS
az storage container create --name tfstate --account-name mystorageaccount

terraform {
backend "azurerm" {
resource_group_name = "myGroup"
storage_account_name = "mystorageaccount"
container_name = "tfstate"
key = "terraform.tfstate"
}
}

Root cause:

Misunderstanding that Terraform backend configuration does not create Azure resources automatically.

#2 Running Terraform apply simultaneously from two machines without waiting for lock release.

Wrong approach:

User A and User B both run 'terraform apply' at the same time on the same backend.

Correct approach:

Wait for the first apply to finish and release the lock before starting another apply.

Root cause:

Not understanding how state locking prevents concurrent conflicting changes.

#3 Not securing access to the Azure Storage account, leaving state files exposed.

Wrong approach:

Assigning Storage Blob Data Contributor role to all users without restrictions.

Correct approach:

Use Azure RBAC to restrict access only to necessary users or service principals, enable encryption and network restrictions.

Root cause:

Underestimating the sensitivity of Terraform state files and Azure security best practices.

Key Takeaways

Terraform state files are the source of truth for infrastructure and must be managed carefully.

Azure Storage backend provides a secure, shared place to store Terraform state, enabling team collaboration.

State locking in Azure Storage backend prevents conflicting changes and protects state integrity.

Proper setup requires creating Azure Storage resources beforehand and configuring access controls.

Migrating state and managing versions are critical for safe backend changes and disaster recovery.