Why data sources query existing infrastructure in Terraform - Why It Works

Introduction

Sometimes you need to use information about resources that already exist outside your current setup. Data sources let you look up this information so you can use it in your Terraform configuration without recreating those resources.

When you want to find the ID of a network that was created manually or by another team.

When you need to get the latest AMI image ID from AWS to launch a server.

When you want to use an existing database instance in your Terraform setup without managing it.

When you need to reference existing security groups to attach to new resources.

When you want to read configuration details from cloud resources created outside Terraform.

Config File - main.tf

main.tf

provider "aws" {
  region = "us-east-1"
}

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
  }
}

resource "aws_instance" "example" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t2.micro"
}

The provider block sets the AWS region.

The data "aws_ami" "ubuntu" block queries AWS for the most recent Ubuntu AMI owned by the official Ubuntu account.

The aws_instance resource uses the AMI ID from the data source to launch a server.

Commands

This command initializes the Terraform working directory, downloads the AWS provider plugin, and prepares Terraform to run.

Terminal

terraform init

Expected Output

Initializing the backend...



Initializing provider plugins...

- Finding latest version of hashicorp/aws...

- Installing hashicorp/aws v4.0.0...

- Installed hashicorp/aws v4.0.0 (signed by HashiCorp)



Terraform has been successfully initialized!



You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure.

This command shows what Terraform will do. It queries the data source to get the latest AMI ID and plans to create an EC2 instance using that AMI.

Terminal

terraform plan

Expected Output

Refreshing Terraform state in-memory prior to plan...



An execution plan has been generated and is shown below.

Resource actions are indicated with the following symbols:

  + create



Terraform will perform the following actions:



  # aws_instance.example will be created

  + resource "aws_instance" "example" {

      + ami                          = "ami-0abcdef1234567890"

      + arn                          = (known after apply)

      + instance_type                = "t2.micro"

      + id                           = (known after apply)

      + tags                         = (known after apply)

    }



Plan: 1 to add, 0 to change, 0 to destroy.

This command applies the planned changes, creating the EC2 instance with the AMI ID obtained from the data source.

Terminal

terraform apply -auto-approve

Expected Output

aws_instance.example: Creating...

aws_instance.example: Still creating... [10s elapsed]

aws_instance.example: Creation complete after 20s [id=i-0123456789abcdef0]



Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

→

-auto-approve - Automatically approve the apply without asking for confirmation

This command displays the current state of the infrastructure Terraform manages, showing the EC2 instance details including the AMI ID used.

Terminal

terraform show

Expected Output

aws_instance.example:

  id = i-0123456789abcdef0

  ami = ami-0abcdef1234567890

  instance_type = t2.micro

  ...

Key Concept

Data sources let Terraform look up existing resources so you can use their details without managing or recreating them.

Common Mistakes

Trying to use a data source without initializing Terraform first.

Terraform needs to download providers and prepare the environment before it can query data sources.

Always run 'terraform init' before 'terraform plan' or 'terraform apply'.

Assuming data sources create or change resources.

Data sources only read existing information; they do not create or modify anything.

Use data sources only to fetch information, and use resources to create or change infrastructure.

Using incorrect filters in data sources causing no results or errors.

If filters do not match any existing resource, Terraform cannot find the data and will error.

Check filter values carefully and test queries to ensure they match existing resources.

Summary

Use 'terraform init' to prepare Terraform and download necessary plugins.

Use data sources in your configuration to query existing infrastructure details.

Run 'terraform plan' to see what Terraform will do using data from data sources.

Apply changes with 'terraform apply' to create resources using queried data.

Use 'terraform show' to verify the current state and resource details.

Practice

1. What is the main reason Terraform uses data sources to query existing infrastructure?

easy

A. To get information about resources created outside Terraform

B. To create new resources automatically

C. To delete unused resources

D. To backup infrastructure state

Solution

1. Step 1: Understand Terraform data sources purpose

   Data sources allow Terraform to read details about resources that exist but were not created by the current Terraform configuration.

2. Step 2: Identify the correct use case

   Since data sources do not create or delete resources, their main use is to fetch information about existing infrastructure.

3. Final Answer:

   To get information about resources created outside Terraform -> Option A

4. Quick Check:

   Data sources read existing resources = A [OK]

Hint: Data sources read info, they don't create or delete [OK]

Common Mistakes:

• Thinking data sources create resources
• Confusing data sources with resource blocks
• Assuming data sources manage resource lifecycle

2. Which of the following is the correct syntax to define a data source in Terraform?

easy

A. output "aws_vpc" "main" { id = "vpc-123456" }

B. resource "aws_vpc" "main" { id = "vpc-123456" }

C. data "aws_vpc" "main" { id = "vpc-123456" }

D. variable "aws_vpc" "main" { id = "vpc-123456" }

Solution

1. Step 1: Identify Terraform block types

   Terraform uses the keyword data to define data sources, resource for resources, output for outputs, and variable for input variables.

2. Step 2: Match syntax for data source

   The correct syntax for a data source starts with data, followed by the provider and resource type, then the name and configuration.

3. Final Answer:

   data "aws_vpc" "main" { id = "vpc-123456" } -> Option C

4. Quick Check:

   Data source syntax starts with 'data' = A [OK]

Hint: Data sources start with 'data' keyword, not 'resource' [OK]

Common Mistakes:

• Using 'resource' instead of 'data' for data sources
• Confusing outputs or variables with data sources
• Incorrect block structure or missing quotes

3. Given this Terraform data source snippet:

data "aws_subnet" "example" {
  filter {
    name   = "tag:Name"
    values = ["public-subnet"]
  }
}

What does data.aws_subnet.example.id return?

medium

A. Creates a new subnet with the name public-subnet

B. Throws an error because filter is invalid

C. Returns a list of all subnet IDs in the AWS account

D. The ID of the subnet tagged with Name=public-subnet

Solution

1. Step 1: Understand the data source filter

   The filter selects the subnet resource with the tag Name equal to "public-subnet".

2. Step 2: Determine the returned value

   Accessing data.aws_subnet.example.id returns the ID of the subnet matching the filter, not creating or listing all subnets.

3. Final Answer:

   The ID of the subnet tagged with Name=public-subnet -> Option D

4. Quick Check:

   Filter selects existing subnet ID = B [OK]

Hint: Filters select existing resources; data source returns their attributes [OK]

Common Mistakes:

• Thinking data source creates resources
• Assuming it returns all subnets without filter
• Misunderstanding filter syntax

4. You wrote this Terraform data source:

data "aws_security_group" "web" {
  name = "web-sg"
}

But Terraform shows an error: "name" is not a valid argument. What is the likely fix?

medium

A. Remove the block entirely; data sources can't query security groups

B. Replace name with filter block to find security group by name

C. Add id argument instead of name

D. Change data to resource

Solution

1. Step 1: Check data source argument rules

   The aws_security_group data source does not accept a direct name argument; it uses filters to find resources.

2. Step 2: Correct usage with filter block

   Use a filter block with name = "group-name" and values = ["web-sg"] to query by name.

3. Final Answer:

   Replace name with filter block to find security group by name -> Option B

4. Quick Check:

   Use filter block for querying by name = D [OK]

Hint: Use filter blocks, not direct name arguments in data sources [OK]

Common Mistakes:

• Using unsupported arguments in data sources
• Confusing resource and data source syntax
• Assuming data sources create or modify resources

5. You want to use a data source to get the latest AMI ID for Ubuntu in your region. Which approach correctly uses a data source to achieve this?

hard

A. data "aws_ami" "ubuntu" {
most_recent = true
owners = ["099720109477"]
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
}
}

B. resource "aws_ami" "ubuntu" {
most_recent = true
owners = ["099720109477"]
}

C. variable "ubuntu_ami" {
default = "ami-123456"
}

D. output "ubuntu_ami" {
value = "ami-123456"
}

Solution

1. Step 1: Identify correct block type for querying existing AMI

   To get info about an existing AMI, use a data block, not resource, variable, or output.

2. Step 2: Verify data source configuration

   The aws_ami data source with most_recent = true, owners, and filter for name pattern correctly fetches the latest Ubuntu AMI ID.

3. Final Answer:

   Use a data source with most_recent, owners, and filter to get latest AMI -> Option A

4. Quick Check:

   Data source with filters fetches latest AMI = C [OK]

Hint: Use data source with filters and most_recent to get latest AMI [OK]

Common Mistakes:

• Using resource block instead of data source
• Hardcoding AMI IDs in variables or outputs
• Not filtering or specifying owners in data source