Creating and switching workspaces in Terraform - Mechanics & Internals

Overview - Creating and switching workspaces

What is it?

In Terraform, workspaces are like separate folders inside the same project that keep your infrastructure states apart. Creating a workspace means making a new space to manage a different set of resources or environments, like testing or production. Switching workspaces lets you move between these spaces easily without mixing their settings or data. This helps you manage multiple environments safely within one Terraform setup.

Why it matters

Without workspaces, managing different environments like development, testing, and production would be risky and confusing because all changes would affect the same infrastructure state. Workspaces solve this by isolating states, preventing accidental changes across environments. This keeps your infrastructure organized, safe, and easier to maintain, especially when working with teams or multiple projects.

Where it fits

Before learning about workspaces, you should understand basic Terraform concepts like configuration files and state files. After mastering workspaces, you can explore advanced topics like remote state backends, state locking, and Terraform modules for reusable infrastructure.

Mental Model

Core Idea

Terraform workspaces are separate containers within one project that keep infrastructure states isolated so you can manage multiple environments safely and easily.

Think of it like...

Imagine a filing cabinet with several drawers. Each drawer holds documents for a different project or purpose. Creating a workspace is like adding a new drawer, and switching workspaces is like opening a different drawer to work on its documents without mixing them up.

┌─────────────────────────────┐
│       Terraform Project      │
│ ┌─────────────┐ ┌─────────┐ │
│ │ Workspace 1 │ │Workspace│ │
│ │ (dev)       │ │  2 (prod)│ │
│ └─────────────┘ └─────────┘ │
│  Separate state files inside │
│  each workspace keep data    │
│  isolated                   │
└─────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What is a Terraform workspace?

Concept: Introducing the idea of workspaces as isolated state containers within one Terraform project.

Terraform uses a state file to keep track of resources it manages. A workspace is a named environment that holds its own state file. By default, Terraform has one workspace called 'default'. Creating new workspaces lets you have multiple states in the same project.

Result

You understand that workspaces separate infrastructure states so you can manage different environments without conflicts.

Understanding that workspaces isolate state files helps you see how Terraform prevents mixing environments in one project.

2

Foundation

How to create a new workspace

Concept: Learning the command to create a new workspace and what happens behind the scenes.

Use the command 'terraform workspace new ' to create a new workspace. This makes a new state file separate from others. For example, 'terraform workspace new staging' creates a workspace called 'staging'.

Result

A new workspace is created with its own state file, ready to manage resources independently.

Knowing how to create workspaces lets you organize your infrastructure into separate environments easily.

3

Intermediate

Switching between workspaces

🤔Before reading on: do you think switching workspaces changes your configuration files or just the state Terraform uses? Commit to your answer.

Concept: Understanding how to move between workspaces and what changes when you do.

Use 'terraform workspace select ' to switch to an existing workspace. This changes which state file Terraform uses but does not change your configuration files. For example, 'terraform workspace select production' switches to the 'production' workspace.

Result

Terraform commands now apply to the selected workspace's state, isolating changes to that environment.

Knowing that switching workspaces changes only the state context prevents confusion about configuration changes.

4

Intermediate

Listing and showing current workspace

🤔Before reading on: do you think Terraform shows all workspaces or only the current one by default? Commit to your answer.

Concept: Commands to view all workspaces and identify which one is active.

Use 'terraform workspace list' to see all workspaces in the project. The current workspace is marked with an asterisk (*). Use 'terraform workspace show' to display the active workspace name.

Result

You can easily check which workspace you are working in and what other workspaces exist.

Being able to list and identify workspaces helps avoid mistakes like applying changes to the wrong environment.

5

Intermediate

Workspace state isolation in practice

🤔Before reading on: do you think resources created in one workspace appear in another workspace's state? Commit to your answer.

Concept: How workspaces keep resource states separate and why this matters.

Each workspace has its own state file. Resources created in one workspace do not appear in another. For example, if you create a VM in 'dev' workspace, it won't exist in 'prod' workspace until you apply there separately.

Result

Infrastructure changes are isolated per workspace, preventing accidental cross-environment impacts.

Understanding state isolation clarifies how workspaces enable safe multi-environment management.

6

Advanced

Limitations and best practices of workspaces

🤔Before reading on: do you think workspaces are suitable for managing completely different projects? Commit to your answer.

Concept: Knowing when to use workspaces and when to choose other methods like separate projects or backends.

Workspaces are best for managing multiple environments of the same project. They are not designed for completely different projects or large-scale multi-team setups. For those, use separate Terraform configurations or remote backends with locking.

Result

You avoid misusing workspaces and choose the right tool for your infrastructure management needs.

Knowing workspace limits prevents scaling problems and state conflicts in complex environments.

7

Expert

How Terraform manages workspace states internally

🤔Before reading on: do you think Terraform stores workspace states as separate files or merges them internally? Commit to your answer.

Concept: Understanding the internal storage and retrieval of workspace states by Terraform.

Terraform stores each workspace's state separately, typically as files named with the workspace name in the local backend or as separate objects in remote backends. When you switch workspaces, Terraform loads the corresponding state file into memory. This isolation is key to preventing state corruption and enabling parallel environment management.

Result

You grasp the internal mechanics that make workspace isolation reliable and efficient.

Understanding internal state management helps troubleshoot workspace issues and optimize backend configurations.

Under the Hood

Terraform keeps a separate state file for each workspace. When you create a workspace, Terraform creates a new state file named after it. Switching workspaces tells Terraform to load and update that specific state file. This separation ensures that resources managed in one workspace do not affect others. The state files can be stored locally or remotely, depending on the backend configuration.

Why designed this way?

Workspaces were designed to provide a simple way to manage multiple environments without duplicating configuration files or projects. By isolating state files, Terraform avoids conflicts and accidental overwrites. Alternatives like separate projects or backends are more complex, so workspaces offer a lightweight solution for common use cases.

┌───────────────┐
│ Terraform CLI │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Workspace API │
└──────┬────────┘
       │
       ▼
┌───────────────┐      ┌───────────────┐
│ State File:   │      │ State File:   │
│ workspace1.tfstate│  │ workspace2.tfstate│
└───────────────┘      └───────────────┘
       │                      │
       ▼                      ▼
┌───────────────┐      ┌───────────────┐
│ Resources in  │      │ Resources in  │
│ workspace1    │      │ workspace2    │
└───────────────┘      └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does switching workspaces change your Terraform configuration files? Commit to yes or no.

Common Belief: Switching workspaces changes the Terraform configuration files to match the workspace.

Reality: Switching workspaces only changes which state file Terraform uses; configuration files remain the same.

Why it matters: Believing configuration changes with workspace switching can cause confusion and errors when applying infrastructure changes.

Quick: Do resources created in one workspace automatically appear in another? Commit to yes or no.

Common Belief: Resources created in one workspace are shared across all workspaces automatically.

Reality: Each workspace has its own isolated state; resources do not appear in other workspaces unless created there.

Why it matters: Assuming shared resources can lead to unexpected missing infrastructure or duplication.

Quick: Are workspaces suitable for managing completely unrelated projects? Commit to yes or no.

Common Belief: Workspaces are a good way to manage multiple unrelated Terraform projects in one place.

Reality: Workspaces are intended for managing multiple environments of the same project, not unrelated projects.

Why it matters: Using workspaces for unrelated projects can cause state confusion and management complexity.

Quick: Does Terraform merge all workspace states into one when applying? Commit to yes or no.

Common Belief: Terraform merges all workspace states into a single state when applying changes.

Reality: Terraform applies changes only to the current workspace's state; states remain separate and isolated.

Why it matters: Misunderstanding this can cause accidental changes to the wrong environment.

Expert Zone

1

Workspace state files are stored separately but share the same configuration, so changes to configuration affect all workspaces unless conditionally handled.

2

Remote backends may handle workspace states differently; some backends store states as separate objects, while others use prefixes or folders.

3

Terraform does not automatically create workspaces for new environments; explicit creation and switching are required to avoid state conflicts.

When NOT to use

Avoid using workspaces to manage completely different projects or large-scale multi-team environments. Instead, use separate Terraform configurations with dedicated remote backends and state locking to prevent conflicts and improve scalability.

Production Patterns

In production, teams use workspaces to manage environments like dev, staging, and prod within the same project. They combine workspaces with remote backends for state storage and use CI/CD pipelines to automate workspace selection and apply steps safely.

Connections

Git Branching

Similar pattern of isolating changes in separate branches or workspaces.

Understanding workspaces is easier when you see them like Git branches, where each branch holds different changes without affecting others until merged.

Virtual Machines

Both isolate environments to prevent interference.

Just like virtual machines isolate operating systems on one physical machine, Terraform workspaces isolate infrastructure states within one project.

Project Management

Workspaces are like separate project folders to keep tasks organized.

Seeing workspaces as project folders helps understand why isolation and clear boundaries prevent confusion and errors.

Common Pitfalls

#1 Applying changes without switching to the correct workspace.

Wrong approach:

terraform apply
# User forgets to switch workspace and applies changes to 'default' instead of 'production'

Correct approach:

terraform workspace select production
terraform apply
# Ensures changes apply to the intended environment

Root cause:

Forgetting that Terraform commands apply to the current workspace's state leads to accidental changes in the wrong environment.

#2 Assuming workspaces isolate configuration files as well as state.

Wrong approach:

# User modifies configuration files expecting changes to affect only one workspace
# But changes apply to all workspaces
resource "aws_instance" "example" {
ami = "ami-123456"
instance_type = "t2.micro"
}

Correct approach:

# Use conditional expressions or separate configurations for workspace-specific settings
variable "env" {
default = terraform.workspace
}
resource "aws_instance" "example" {
ami = var.env == "production" ? "ami-prod" : "ami-dev"
instance_type = "t2.micro"
}

Root cause:

Misunderstanding that workspaces only isolate state, not configuration, causes unintended resource changes across environments.

#3 Using workspaces to manage unrelated projects.

Wrong approach:

terraform workspace new projectA
terraform workspace new projectB
# Both projects share the same configuration files

Correct approach:

# Create separate Terraform projects with their own configurations and backends
/projectA/main.tf
/projectB/main.tf

Root cause:

Confusing workspaces as project containers leads to state conflicts and management difficulties.

Key Takeaways

Terraform workspaces isolate infrastructure states within the same project to safely manage multiple environments.

Creating and switching workspaces changes which state file Terraform uses, but does not alter configuration files.

Each workspace has its own independent state, so resources in one workspace do not affect others.

Workspaces are best for managing multiple environments of the same project, not for unrelated projects.

Understanding workspace mechanics helps prevent common mistakes like applying changes to the wrong environment.