CLI scripting basics in AWS - Deep Dive

Overview - CLI scripting basics

What is it?

CLI scripting basics means using simple text commands to control cloud services like AWS. Instead of clicking buttons, you type commands in a terminal or script file to automate tasks. This helps you manage resources faster and repeat actions without mistakes. Scripts are like recipes that tell the computer exactly what to do step-by-step.

Why it matters

Without CLI scripting, managing cloud resources would be slow and error-prone because you'd have to do everything manually. Scripts save time, reduce errors, and let you repeat complex tasks easily. This is important when you have many resources or need to do the same setup multiple times. It makes cloud work reliable and efficient.

Where it fits

Before learning CLI scripting, you should know basic cloud concepts and how to use the AWS CLI tool. After this, you can learn advanced scripting with loops, conditions, and integrating scripts into automation pipelines like CI/CD.

Mental Model

Core Idea

CLI scripting is writing step-by-step instructions in text form to control cloud services automatically.

Think of it like...

It's like writing a shopping list and recipe before cooking, so you don't forget ingredients or steps and can make the dish perfectly every time.

┌───────────────┐
│ Start Script  │
└──────┬────────┘
       │
┌──────▼────────┐
│ Run CLI Cmd 1 │
└──────┬────────┘
       │
┌──────▼────────┐
│ Run CLI Cmd 2 │
└──────┬────────┘
       │
┌──────▼────────┐
│   End Script  │
└───────────────┘

Build-Up - 7 Steps

1

Foundation

What is CLI and AWS CLI

Concept: Introduce the command line interface and AWS CLI tool basics.

The CLI is a way to type commands to your computer instead of clicking. AWS CLI is a tool that lets you type commands to control AWS cloud services. You install it on your computer and use simple commands like 'aws s3 ls' to list storage buckets.

Result

You can run commands to see AWS resources from your terminal.

Understanding the CLI and AWS CLI tool is the first step to automating cloud tasks without using the web console.

2

Foundation

Writing Your First Script File

Concept: Learn how to write a simple script file to run multiple AWS CLI commands.

A script file is a text file with commands listed one after another. For example, create a file named 'script.sh' with: aws s3 ls aws ec2 describe-instances Then run it in the terminal with 'bash script.sh'. This runs both commands in order.

Result

Multiple AWS commands run automatically one after another.

Scripts let you save and repeat command sequences, avoiding manual typing each time.

3

Intermediate

Using Variables in Scripts

🤔Before reading on: do you think variables in scripts can store AWS resource names or IDs? Commit to your answer.

Concept: Introduce variables to store values like resource names for reuse in scripts.

Variables hold information you want to reuse. In bash scripts, you write: BUCKET_NAME=my-bucket-123 aws s3 ls s3://$BUCKET_NAME This lists the contents of the bucket stored in BUCKET_NAME. Variables make scripts flexible and easier to update.

Result

Scripts can use variables to run commands on different resources without changing many lines.

Knowing variables lets you write adaptable scripts that work with different inputs easily.

4

Intermediate

Adding Simple Logic with Conditions

🤔Before reading on: can scripts check if a resource exists before acting? Commit to yes or no.

Concept: Show how to use if-statements to make decisions in scripts.

Scripts can check conditions. For example: if aws s3 ls s3://my-bucket-123 2>&1 | grep -q 'NoSuchBucket'; then echo 'Bucket does not exist' else echo 'Bucket exists' fi This checks if a bucket exists before continuing.

Result

Scripts can avoid errors by checking conditions before running commands.

Adding logic makes scripts smarter and safer to run in real environments.

5

Intermediate

Looping Over Multiple Resources

🤔Before reading on: do you think loops can automate commands over many AWS resources? Commit to your answer.

Concept: Introduce loops to repeat commands for multiple items.

Loops run commands repeatedly. Example: for bucket in bucket1 bucket2 bucket3; do aws s3 ls s3://$bucket done This lists contents of three buckets automatically.

Result

Scripts can handle many resources without copying commands multiple times.

Loops save time and reduce mistakes when managing many cloud resources.

6

Advanced

Handling Errors Gracefully in Scripts

🤔Before reading on: do you think scripts stop automatically on errors or keep running? Commit to your answer.

Concept: Teach how to detect and respond to errors in scripts.

By default, scripts keep running even if a command fails. Use 'set -e' at the top to stop on errors: #!/bin/bash set -e aws s3 ls s3://nonexistent-bucket echo 'This will not run if the above fails' You can also check exit codes with '$?'.

Result

Scripts stop or handle errors to avoid unwanted actions after failures.

Proper error handling prevents scripts from causing damage or confusion in production.

7

Expert

Combining CLI Scripts with AWS IAM Roles

🤔Before reading on: do you think scripts can securely access AWS without embedding passwords? Commit to yes or no.

Concept: Explain how to use IAM roles and profiles to run scripts securely without hardcoding credentials.

Instead of putting passwords in scripts, use AWS IAM roles or named profiles. For example, configure a profile: aws configure --profile myprofile Then run commands with: aws s3 ls --profile myprofile Or run scripts on EC2 instances with attached IAM roles that grant permissions automatically.

Result

Scripts run securely with proper permissions, avoiding credential leaks.

Understanding secure access methods is critical for safe automation in real cloud environments.

Under the Hood

When you run an AWS CLI command, the CLI tool sends a request over the internet to AWS servers using your credentials. The script is just a text file that the shell reads line by line, executing each command. Variables and logic are handled by the shell before commands run. IAM roles provide temporary credentials managed by AWS, so scripts don't need permanent passwords.

Why designed this way?

CLI scripting evolved to automate repetitive tasks and reduce human error. AWS designed the CLI to be simple and script-friendly, using standard shell syntax. IAM roles were created to improve security by avoiding hardcoded credentials, following best practices in cloud security.

┌─────────────┐      ┌───────────────┐      ┌───────────────┐
│ Script File │─────▶│ Shell Interpreter│────▶│ AWS CLI Tool  │
└─────────────┘      └───────────────┘      └──────┬────────┘
                                                    │
                                                    ▼
                                            ┌───────────────┐
                                            │ AWS Cloud API │
                                            └───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think CLI scripts always run faster than manual commands? Commit to yes or no.

Common Belief: Scripts always run faster than typing commands manually.

Reality: Scripts save time overall but individual commands run at the same speed; the main gain is automation and repeatability, not raw speed.

Why it matters: Expecting scripts to be faster in every way can lead to frustration; the real benefit is consistency and error reduction.

Quick: Can you safely store AWS passwords directly in scripts? Commit to yes or no.

Common Belief: It's okay to put AWS passwords or keys directly inside scripts for convenience.

Reality: Storing credentials in scripts is insecure and risks leaks; best practice is to use IAM roles or environment variables.

Why it matters: Credential leaks can lead to unauthorized access and costly security breaches.

Quick: Do you think scripts automatically handle all AWS errors? Commit to yes or no.

Common Belief: Scripts automatically stop or fix errors without extra coding.

Reality: Scripts keep running unless you add error handling; ignoring errors can cause bigger problems.

Why it matters: Not handling errors can cause scripts to perform unwanted actions or leave resources in bad states.

Quick: Do you think variables in scripts can store complex AWS objects directly? Commit to yes or no.

Common Belief: You can store AWS objects like instances or buckets directly in script variables as full objects.

Reality: Variables store text or numbers; to handle complex data, scripts parse JSON output or use specialized tools.

Why it matters: Misunderstanding this leads to script errors and confusion when processing AWS data.

Expert Zone

1

Scripts can be combined with AWS CloudFormation or Terraform for hybrid automation approaches.

2

Using AWS CLI's JSON output with tools like jq allows powerful data filtering inside scripts.

3

Scripts run on different shells (bash, zsh, PowerShell) behave differently; knowing shell specifics avoids bugs.

When NOT to use

Avoid CLI scripting for very large or complex infrastructure setups where infrastructure as code tools like Terraform or CloudFormation provide better management, versioning, and safety.

Production Patterns

Professionals use CLI scripts for quick fixes, small automation tasks, or integrating AWS commands into CI/CD pipelines. Scripts often run on developer machines, build servers, or AWS Lambda functions with proper IAM roles.

Connections

Infrastructure as Code (IaC)

CLI scripting builds on basic automation, while IaC tools manage entire infrastructure declaratively.

Understanding CLI scripting helps grasp how IaC tools execute commands under the hood and why automation matters.

Unix Shell Scripting

CLI scripting uses shell scripting concepts like variables, loops, and conditions.

Mastering shell scripting improves AWS CLI script quality and flexibility.

Project Management Workflows

Scripts automate repetitive tasks in project workflows, similar to how checklists improve task consistency.

Seeing scripting as workflow automation connects cloud tasks to everyday productivity methods.

Common Pitfalls

#1 Hardcoding sensitive credentials in scripts.

Wrong approach:

aws configure set aws_access_key_id ABC123
aws configure set aws_secret_access_key XYZ789

# Using these directly in scripts like:
aws s3 ls --access-key ABC123 --secret-key XYZ789

Correct approach:

Use IAM roles or environment variables:

export AWS_PROFILE=myprofile
aws s3 ls

Or attach IAM role to EC2 instance running the script.

Root cause:

Lack of understanding of secure credential management leads to risky practices.

#2 Ignoring command errors and continuing script execution.

Wrong approach:

#!/bin/bash
aws s3 ls s3://nonexistent-bucket
# Script continues even if above fails
aws ec2 describe-instances

Correct approach:

#!/bin/bash
set -e
aws s3 ls s3://nonexistent-bucket
aws ec2 describe-instances

Root cause:

Not knowing how to stop scripts on errors causes unintended consequences.

#3 Using incorrect variable syntax causing commands to fail.

Wrong approach:

BUCKET=my-bucket
aws s3 ls s3://BUCKET
# This tries to list bucket named 'BUCKET' literally

Correct approach:

BUCKET=my-bucket
aws s3 ls s3://$BUCKET
# Correctly uses variable value

Root cause:

Misunderstanding shell variable syntax leads to wrong command execution.

Key Takeaways

CLI scripting automates cloud tasks by running sequences of commands in text files.

Using variables, conditions, and loops makes scripts flexible and powerful.

Proper error handling and secure credential management are essential for safe scripts.

Scripts are a foundation for cloud automation but have limits compared to full infrastructure as code tools.

Mastering CLI scripting improves efficiency, reduces errors, and prepares you for advanced cloud automation.

Practice

1. What does the AWS CLI command aws s3 ls do?

easy

A. Deletes all S3 buckets in your AWS account

B. Creates a new S3 bucket

C. Lists all S3 buckets in your AWS account

D. Uploads a file to an S3 bucket

Solution

1. Step 1: Understand the command structure

   The command uses aws s3 which relates to the S3 service, and ls which means list.

2. Step 2: Interpret the command action

   Listing in S3 context means showing all buckets or objects. Without extra parameters, it lists all buckets.

3. Final Answer:

   Lists all S3 buckets in your AWS account -> Option C

4. Quick Check:

   Command aws s3 ls lists buckets [OK]

Hint: Remember: 'ls' means list, so it shows existing resources [OK]

Common Mistakes:

• Confusing 'ls' with 'rm' (delete)
• Thinking it creates resources
• Assuming it uploads files

2. Which of the following is the correct syntax to create an EC2 instance using AWS CLI?

easy

A. aws ec2 run-instances --image-id ami-12345678 --count 1

B. aws ec2 create-instance --image-id ami-12345678 --count 1

C. aws ec2 start-instance --image-id ami-12345678 --count 1

D. aws ec2 launch-instance --image-id ami-12345678 --count 1

Solution

1. Step 1: Identify the correct AWS CLI command for EC2 instance creation

   The official command to create EC2 instances is run-instances.

2. Step 2: Check the syntax correctness

   aws ec2 run-instances --image-id ami-12345678 --count 1 uses aws ec2 run-instances --image-id ami-12345678 --count 1, which is the correct syntax to launch one instance.

3. Final Answer:

   aws ec2 run-instances --image-id ami-12345678 --count 1 -> Option A

4. Quick Check:

   EC2 instance creation uses 'run-instances' command [OK]

Hint: Use 'run-instances' to launch EC2, not 'create-instance' [OK]

Common Mistakes:

• Using 'create-instance' which is invalid
• Confusing 'start-instance' with creation
• Using 'launch-instance' which is not a valid command

3. What will be the output of this AWS CLI command?

aws s3api list-buckets --query 'Buckets[].Name' --output json

medium

A. A JSON array of bucket names

B. A list of bucket creation dates

C. An error because of wrong syntax

D. A plain text list of bucket names

Solution

1. Step 1: Understand the command components

   The command uses list-buckets to get all buckets, with a query to extract only the bucket names.

2. Step 2: Analyze the output format

   The --output json option formats the result as JSON, so the output is a JSON array of bucket names.

3. Final Answer:

   A JSON array of bucket names -> Option A

4. Quick Check:

   Query filters names, output json formats as JSON array [OK]

Hint: Query filters data, output json formats it as JSON [OK]

Common Mistakes:

• Expecting plain text instead of JSON
• Confusing bucket names with creation dates
• Assuming syntax error due to query

4. You run this command to delete an S3 bucket:

aws s3 rb s3://my-bucket

But it fails with an error. What is the most likely cause?

medium

A. You need to use aws s3 rm instead

B. The bucket is not empty

C. The bucket name is invalid

D. The AWS CLI is not installed

Solution

1. Step 1: Understand the command purpose

   aws s3 rb removes (deletes) an S3 bucket, but only if it is empty.

2. Step 2: Identify the common error cause

   If the bucket contains files, the command fails. You must delete all objects first.

3. Final Answer:

   The bucket is not empty -> Option B

4. Quick Check:

   Bucket must be empty before removal [OK]

Hint: Empty bucket before deleting with 'rb' command [OK]

Common Mistakes:

• Trying to delete non-empty bucket directly
• Confusing 'rm' (remove objects) with 'rb' (remove bucket)
• Assuming AWS CLI is not installed without checking

5. You want to write a script that lists all EC2 instances and then stops each one. Which AWS CLI commands should you combine in the script?

hard

A. Use aws ec2 show-instances to list, then aws ec2 power-off-instances to stop

B. Use aws ec2 list-instances to list, then aws ec2 terminate-instances to stop

C. Use aws ec2 get-instances to list, then aws ec2 shutdown-instances to stop

D. Use aws ec2 describe-instances to list, then aws ec2 stop-instances --instance-ids to stop

Solution

1. Step 1: Identify the correct command to list EC2 instances

   The correct command to list instances is describe-instances.

2. Step 2: Identify the correct command to stop instances

   To stop instances, use stop-instances with the instance IDs.

3. Step 3: Combine commands in script logic

   First list instances to get IDs, then stop them using those IDs.

4. Final Answer:

   Use aws ec2 describe-instances to list, then aws ec2 stop-instances --instance-ids to stop -> Option D

5. Quick Check:

   List with describe, stop with stop-instances [OK]

Hint: List with 'describe-instances', stop with 'stop-instances' [OK]

Common Mistakes:

• Using non-existent commands like 'list-instances'
• Confusing 'terminate' with 'stop' (termination deletes)
• Using wrong commands that cause errors