Stack drift detection in AWS - Deep Dive

Overview - Stack drift detection

What is it?

Stack drift detection is a way to find out if the actual resources in your cloud setup have changed from what you originally planned. It checks if anything was added, removed, or changed outside of your main setup instructions. This helps keep your cloud environment reliable and predictable. It is like a safety check to catch unexpected changes.

Why it matters

Without drift detection, your cloud setup can slowly change without you noticing. This can cause problems like security risks, broken applications, or unexpected costs. Drift detection helps you spot these changes early so you can fix them before they cause trouble. It keeps your cloud environment trustworthy and easier to manage.

Where it fits

Before learning drift detection, you should understand cloud infrastructure basics and how to define your setup using tools like AWS CloudFormation. After drift detection, you can learn about automated monitoring, compliance checks, and remediation strategies to keep your cloud environment healthy.

Mental Model

Core Idea

Stack drift detection compares your cloud setup plan with the real setup to find any differences caused by changes outside your main instructions.

Think of it like...

Imagine you have a recipe to bake a cake, but someone secretly changes the ingredients or steps while you’re not looking. Drift detection is like tasting the cake to see if it matches the original recipe or if something was changed.

┌───────────────────────────────┐
│       Cloud Setup Plan         │
│  (Your original instructions) │
└──────────────┬────────────────┘
               │
               │ Compare
               ▼
┌───────────────────────────────┐
│      Actual Cloud Setup        │
│ (What really exists in cloud)  │
└──────────────┬────────────────┘
               │
               │ Detect Differences
               ▼
┌───────────────────────────────┐
│        Drift Report            │
│ (Shows added, removed, changed│
│        resources)             │
└───────────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What is a Cloud Stack

Concept: Introduce the idea of a cloud stack as a group of resources managed together.

A cloud stack is like a set of building blocks that make up an application or service in the cloud. For example, it can include servers, databases, and networks all defined in one place. Tools like AWS CloudFormation let you write down what you want in your stack so the cloud creates it for you.

Result

You understand that a stack is a planned collection of cloud resources managed as one unit.

Knowing what a stack is helps you see why tracking changes to it matters for keeping your cloud setup consistent.

2

Foundation

How Cloud Stacks Change Over Time

Concept: Explain that cloud stacks can change both through planned updates and unexpected manual changes.

After creating a stack, you might update it by changing the instructions and applying them. But sometimes, people or systems make changes directly in the cloud without updating the instructions. These unplanned changes cause the stack to drift from its original plan.

Result

You realize that cloud stacks can become different from their original design if changes happen outside the main instructions.

Understanding that changes can happen outside your control sets the stage for why drift detection is needed.

3

Intermediate

What is Stack Drift Detection

🤔Before reading on: do you think drift detection finds only missing resources or all types of changes? Commit to your answer.

Concept: Introduce drift detection as a process that finds all differences between the planned and actual stack state.

Stack drift detection compares each resource in your planned stack with what actually exists in the cloud. It checks if resources were added, removed, or modified. AWS CloudFormation provides a drift detection feature that scans your stack and reports any differences.

Result

You can identify any unexpected changes in your cloud stack, not just missing parts but also modifications and additions.

Knowing that drift detection covers all types of changes helps you trust it as a complete check for your cloud setup.

4

Intermediate

How to Run Drift Detection in AWS

🤔Before reading on: do you think drift detection runs automatically or must be started manually? Commit to your answer.

Concept: Teach the practical steps to start drift detection using AWS tools.

In AWS, you can start drift detection from the CloudFormation console, CLI, or API. When started, AWS compares the stack's template and resource properties with the actual resources. After the scan, it shows a report listing any drifted resources and details about the differences.

Result

You know how to trigger drift detection and read its results to find out if your stack has drifted.

Understanding how to run drift detection empowers you to keep your cloud environment consistent and secure.

5

Intermediate

Types of Drift Detected

🤔Before reading on: do you think drift detection can find changes in resource tags or only core settings? Commit to your answer.

Concept: Explain the kinds of changes drift detection can find, including properties and tags.

Drift detection checks resource properties like configuration settings and also resource metadata like tags. It can detect if a resource was deleted, replaced, or modified. However, some resource types or properties might not be supported for drift detection, so not all changes are always found.

Result

You understand the scope and limits of what drift detection can find in your stack.

Knowing what drift detection covers helps you interpret its reports correctly and plan additional checks if needed.

6

Advanced

Handling Drift in Production Environments

🤔Before reading on: do you think drift detection fixes drift automatically or requires manual action? Commit to your answer.

Concept: Discuss best practices for managing drift in real cloud environments.

Drift detection only reports differences; it does not fix them automatically. In production, teams use drift detection regularly to monitor stacks. When drift is found, they investigate and decide whether to update the stack template, revert manual changes, or redeploy resources. Automating remediation requires extra tools or scripts.

Result

You see how drift detection fits into a larger process of cloud management and governance.

Understanding that drift detection is a detection tool, not a fix, clarifies its role and helps you design proper response workflows.

7

Expert

Limitations and Surprises of Drift Detection

🤔Before reading on: do you think drift detection always finds all changes instantly? Commit to your answer.

Concept: Reveal subtle limitations and behaviors of drift detection that can surprise even experienced users.

Drift detection depends on resource type support and may not detect changes in some resources or properties. It also only detects drift at the time of the scan; changes after require new scans. Some changes, like external dependencies or out-of-band effects, are invisible to drift detection. Additionally, drift detection can take time on large stacks and may impact API limits.

Result

You gain a realistic understanding of drift detection’s boundaries and operational considerations.

Knowing these limits prevents overreliance on drift detection and encourages complementary monitoring strategies.

Under the Hood

Stack drift detection works by querying the current state of each resource in the cloud and comparing it to the expected state defined in the stack template and parameters. AWS CloudFormation uses APIs to fetch resource properties and metadata, then performs a property-by-property comparison. Differences are recorded as drift. This process involves mapping resource identifiers, handling resource types differently, and managing API rate limits.

Why designed this way?

Drift detection was designed to help users maintain infrastructure as code integrity without manual audits. It balances thoroughness with performance by focusing on supported resource types and key properties. Alternatives like continuous monitoring or manual checks were less scalable or error-prone. The design trades off full coverage for practical usability and integration with existing CloudFormation workflows.

┌───────────────────────────────┐
│     Stack Template & Params   │
└──────────────┬────────────────┘
               │
               │ Defines expected state
               ▼
┌───────────────────────────────┐
│   CloudFormation Drift Engine  │
│  ┌─────────────────────────┐  │
│  │ Query Actual Resources  │  │
│  └────────────┬────────────┘  │
│               │               │
│  ┌────────────▼────────────┐  │
│  │ Compare Expected vs     │  │
│  │ Actual Properties       │  │
│  └────────────┬────────────┘  │
│               │               │
│  ┌────────────▼────────────┐  │
│  │ Generate Drift Report   │  │
│  └─────────────────────────┘  │
└───────────────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does drift detection automatically fix detected drift? Commit to yes or no.

Common Belief: Drift detection automatically corrects any differences it finds in the stack.

Reality: Drift detection only reports differences; it does not change or fix any resources automatically.

Why it matters: Assuming automatic fixes can lead to ignoring drift reports and leaving your cloud environment inconsistent and vulnerable.

Quick: Can drift detection find changes in all AWS resource types? Commit to yes or no.

Common Belief: Drift detection works perfectly on every AWS resource type and property.

Reality: Some resource types or properties are not supported by drift detection, so some changes may go unnoticed.

Why it matters: Relying solely on drift detection can give a false sense of security if unsupported resources drift without detection.

Quick: Does drift detection monitor changes continuously in real time? Commit to yes or no.

Common Belief: Drift detection continuously watches your stack and alerts you instantly on any change.

Reality: Drift detection runs only when manually started or scheduled; it does not provide real-time monitoring.

Why it matters: Expecting real-time alerts can cause delays in detecting drift and responding to issues.

Quick: If a resource is changed outside the stack, will drift detection always find it immediately? Commit to yes or no.

Common Belief: Any out-of-stack change is instantly detected by drift detection.

Reality: Drift detection only finds changes present at the time of the scan; changes after require new scans.

Why it matters: Believing in instant detection can lead to missed drift if scans are infrequent.

Expert Zone

1

Drift detection results can be affected by eventual consistency in cloud APIs, causing temporary false positives or negatives.

2

Some resource properties are considered 'read-only' and changes to them do not cause drift, which can confuse users expecting full detection.

3

Stack drift detection does not detect changes in resources outside the stack even if they affect stack behavior, requiring complementary monitoring.

When NOT to use

Do not rely on drift detection as your only compliance or security tool. For real-time monitoring, use AWS Config or CloudTrail. For unsupported resources, consider custom scripts or third-party tools. Drift detection is also not suitable for very large stacks requiring continuous monitoring due to scan time and API limits.

Production Patterns

In production, teams schedule regular drift detection scans as part of their deployment pipelines or monitoring routines. They integrate drift reports with ticketing systems for manual review. Some automate remediation for common drift cases using infrastructure as code updates or rollback scripts. Drift detection is combined with AWS Config for comprehensive compliance.

Connections

Infrastructure as Code (IaC)

Builds-on

Understanding drift detection deepens your grasp of IaC by showing how to verify that your code matches reality, reinforcing the principle of treating infrastructure like software.

Version Control Systems

Similar pattern

Drift detection is like a version control diff tool but for cloud resources, comparing desired state with actual state to find changes, highlighting the importance of tracking changes in any system.

Quality Control in Manufacturing

Analogous process

Drift detection parallels quality control checks in factories, where products are inspected to ensure they match design specs, showing how verification is key in both physical and digital production.

Common Pitfalls

#1 Ignoring drift detection reports assuming no drift exists.

Wrong approach:

/* No drift detection run or ignoring drift report */

Correct approach:

aws cloudformation detect-stack-drift --stack-name MyStack
aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id

Root cause:

Misunderstanding that drift detection must be actively run and reviewed to be effective.

#2 Expecting drift detection to fix drift automatically.

Wrong approach:

/* Running drift detection and expecting AWS to fix resources */

Correct approach:

Review drift report and update CloudFormation template or manually fix resources as needed.

Root cause:

Confusing detection with remediation; drift detection only reports differences.

#3 Assuming drift detection covers all resource types and properties.

Wrong approach:

/* Trusting drift detection report as complete without verifying resource support */

Correct approach:

Check AWS documentation for supported resource types and supplement with other monitoring tools.

Root cause:

Lack of awareness about drift detection limitations.

Key Takeaways

Stack drift detection helps find differences between your planned cloud setup and what actually exists, catching unexpected changes.

It does not fix drift automatically; it only reports differences for you to review and act upon.

Drift detection supports many but not all resource types and properties, so it should be part of a broader monitoring strategy.

Running drift detection regularly helps maintain cloud environment consistency, security, and reliability.

Understanding drift detection’s limits and operation helps you use it effectively and avoid common mistakes.

Practice

1.

What is the main purpose of stack drift detection in AWS CloudFormation?

easy

A. To find manual changes made outside of CloudFormation templates

B. To automatically fix errors in CloudFormation templates

C. To create new stacks from existing templates

D. To monitor network traffic between AWS resources

Solution

1. Step 1: Understand stack drift detection concept

   Stack drift detection checks if resources in a stack have been changed manually or outside CloudFormation management.

2. Step 2: Identify the main purpose

   The main goal is to find these manual or unexpected changes to keep infrastructure consistent.

3. Final Answer:

   To find manual changes made outside of CloudFormation templates -> Option A

4. Quick Check:

   Stack drift detection = find manual changes [OK]

Hint: Stack drift detection finds changes outside CloudFormation [OK]

Common Mistakes:

• Thinking it fixes errors automatically
• Confusing it with stack creation
• Assuming it monitors network traffic

2.

Which AWS CLI command starts drift detection for a CloudFormation stack named MyAppStack?

easy

A. aws cloudformation begin-drift-check --name MyAppStack

B. aws cloudformation detect-stack-drift --stack-name MyAppStack

C. aws cloudformation start-drift-detection --stack MyAppStack

D. aws cloudformation check-drift --stack-name MyAppStack

Solution

1. Step 1: Recall AWS CLI syntax for drift detection

   The correct command to start drift detection is detect-stack-drift with the --stack-name option.

2. Step 2: Match the command with the stack name

   aws cloudformation detect-stack-drift --stack-name MyAppStack uses the exact syntax and correct parameter for the stack name.

3. Final Answer:

   aws cloudformation detect-stack-drift --stack-name MyAppStack -> Option B

4. Quick Check:

   Detect drift command = detect-stack-drift [OK]

Hint: Use 'detect-stack-drift' with '--stack-name' to start detection [OK]

Common Mistakes:

• Using incorrect command verbs like start or begin
• Wrong parameter names like --stack or --name
• Mixing up command order or syntax

3.

After running aws cloudformation detect-stack-drift --stack-name MyStack, which command shows the drift detection status and results?

medium

A. aws cloudformation show-drift-results --stack-name MyStack

B. aws cloudformation get-stack-drift-status --stack-name MyStack

C. aws cloudformation list-stack-drift-detection --stack-name MyStack

D. aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id

Solution

1. Step 1: Identify the command to check drift detection status

   The AWS CLI command to check drift detection status is describe-stack-drift-detection-status.

2. Step 2: Confirm the correct parameter usage

   It requires the --stack-drift-detection-id parameter to specify the detection ID, not --stack-name.

3. Final Answer:

   aws cloudformation describe-stack-drift-detection-status --stack-drift-detection-id <detection-id> -> Option D

4. Quick Check:

   Check drift status = describe-stack-drift-detection-status [OK]

Hint: Use 'describe-stack-drift-detection-status' with detection ID to get drift results [OK]

Common Mistakes:

• Using non-existent commands like get-stack-drift-status
• Confusing list commands with status commands
• Wrong parameter names

4.

You ran aws cloudformation detect-stack-drift --stack-name TestStack but got an error: Stack with id TestStack does not exist. What is the most likely cause?

medium

A. The stack name is misspelled or the stack does not exist

B. Drift detection is not supported for this stack type

C. You need to add --region parameter to the command

D. The AWS CLI version is outdated and cannot run drift detection

Solution

1. Step 1: Analyze the error message

   The error says the stack with the given name does not exist, indicating a wrong name or missing stack.

2. Step 2: Consider other causes

   While region or CLI version issues can cause errors, this specific message points to a missing or misspelled stack name.

3. Final Answer:

   The stack name is misspelled or the stack does not exist -> Option A

4. Quick Check:

   Error about stack id means wrong or missing stack [OK]

Hint: Check stack name spelling and existence first [OK]

Common Mistakes:

• Assuming region parameter is always required
• Blaming CLI version without checking stack
• Ignoring the exact error message

5.

You want to ensure your CloudFormation stack stays consistent with its template. Which approach best uses stack drift detection to maintain this?

A) Run drift detection regularly and update stack if drift is detected.
B) Disable drift detection and manually check resources.
C) Use drift detection only after stack deletion.
D) Ignore drift detection and rely on CloudWatch alarms.

hard

A. Use drift detection only after stack deletion

B. Disable drift detection and manually check resources

C. Run drift detection regularly and update stack if drift is detected

D. Ignore drift detection and rely on CloudWatch alarms

Solution

1. Step 1: Understand drift detection best practice

   Regularly running drift detection helps find manual changes early and keeps the stack consistent.

2. Step 2: Evaluate options

   Run drift detection regularly and update stack if drift is detected uses drift detection proactively. Options B, C, and D either ignore drift detection or use it incorrectly.

3. Final Answer:

   Run drift detection regularly and update stack if drift is detected -> Option C

4. Quick Check:

   Regular drift detection = best practice [OK]

Hint: Run drift detection regularly to catch changes early [OK]

Common Mistakes:

• Ignoring drift detection completely
• Checking drift only after deletion
• Relying solely on CloudWatch for drift