CloudWatch Logs in AWS - Mini Project: Build & Apply

Basic CloudWatch Logs Setup

📖 Scenario: You are setting up a simple logging system for a web application running on AWS. You want to collect logs from your application and store them in AWS CloudWatch Logs for monitoring and troubleshooting.

🎯 Goal: Create a CloudWatch Logs group and stream, configure a log event, and complete the setup to send logs to CloudWatch Logs.

📋 What You'll Learn

Create a CloudWatch Logs group named MyAppLogGroup

Create a CloudWatch Logs stream named MyAppLogStream inside the log group

Create a log event with a timestamp and message

Complete the configuration to put the log event into the log stream

💡 Why This Matters

🌍 Real World

CloudWatch Logs is used to collect and monitor logs from applications and infrastructure in AWS, helping teams troubleshoot and analyze system behavior.

💼 Career

Understanding how to configure CloudWatch Logs is essential for cloud engineers and developers working with AWS to maintain application health and security.

1

Create a CloudWatch Logs group

Create a variable called log_group_name and set it to the string 'MyAppLogGroup'.

AWS

# Create a variable called log_group_name and set it to 'MyAppLogGroup'
# Your code here

Hint

Use a simple string assignment to create the log group name.

2

Create a CloudWatch Logs stream

Create a variable called log_stream_name and set it to the string 'MyAppLogStream'.

AWS

log_group_name = 'MyAppLogGroup'
# Create a variable called log_stream_name and set it to 'MyAppLogStream'
# Your code here

Hint

Assign the log stream name as a string variable.

3

Create a log event

Create a dictionary called log_event with keys 'timestamp' and 'message'. Set 'timestamp' to 1680000000000 and 'message' to 'User login successful'.

AWS

log_group_name = 'MyAppLogGroup'
log_stream_name = 'MyAppLogStream'
# Create a dictionary called log_event with keys 'timestamp' and 'message'
# Your code here

Hint

Use a dictionary with the exact keys and values as shown.

4

Complete the CloudWatch Logs configuration

Create a dictionary called cloudwatch_config with keys 'logGroupName', 'logStreamName', and 'logEvents'. Set 'logGroupName' to log_group_name, 'logStreamName' to log_stream_name, and 'logEvents' to a list containing log_event.

AWS

log_group_name = 'MyAppLogGroup'
log_stream_name = 'MyAppLogStream'
log_event = {'timestamp': 1680000000000, 'message': 'User login successful'}
# Create a dictionary called cloudwatch_config with keys 'logGroupName', 'logStreamName', and 'logEvents'
# Your code here

Hint

Combine the variables into a configuration dictionary for CloudWatch Logs.

Practice

1. What is the primary purpose of a CloudWatch Logs log group?

easy

A. To act as a container for multiple log streams

B. To store individual log events directly

C. To trigger alarms based on log data

D. To archive logs permanently

Solution

1. Step 1: Understand log group role

   A log group is a container that holds multiple log streams, organizing logs by application or resource.

2. Step 2: Differentiate log streams and groups

   Log streams are individual sequences of log events within a log group, not the group itself.

3. Final Answer:

   To act as a container for multiple log streams -> Option A

4. Quick Check:

   Log group = container for streams [OK]

Hint: Log groups hold streams; streams hold logs [OK]

Common Mistakes:

• Confusing log groups with log streams
• Thinking logs are stored directly in groups
• Assuming groups trigger alarms directly

2. Which of the following is the correct AWS CLI command to create a new CloudWatch Logs log group named MyAppLogs?

easy

A. aws cloudwatch create-log-group --name MyAppLogs

B. aws logs create-log-group --log-group-name MyAppLogs

C. aws logs new-log-group --group-name MyAppLogs

D. aws logs add-log-group --name MyAppLogs

Solution

1. Step 1: Identify correct AWS CLI service and command

   The correct service is 'logs' and the command to create a log group is 'create-log-group'.

2. Step 2: Check command syntax

   The parameter for the log group name is '--log-group-name', matching aws logs create-log-group --log-group-name MyAppLogs exactly.

3. Final Answer:

   aws logs create-log-group --log-group-name MyAppLogs -> Option B

4. Quick Check:

   Correct CLI syntax = aws logs create-log-group --log-group-name MyAppLogs [OK]

Hint: Use 'aws logs create-log-group' with --log-group-name [OK]

Common Mistakes:

• Using wrong service name like 'cloudwatch' instead of 'logs'
• Incorrect command verbs like 'new-log-group'
• Wrong parameter names like '--name'

3. Given the following AWS CLI command, what will be the retention period set for the log group AppLogs?

aws logs put-retention-policy --log-group-name AppLogs --retention-in-days 14

medium

A. Logs will be kept for 7 days

B. Logs will be kept indefinitely

C. Logs will be kept for 14 days

D. Logs will be deleted immediately

Solution

1. Step 1: Understand the put-retention-policy command

   This command sets how many days logs are retained before deletion.

2. Step 2: Interpret the retention-in-days value

   The parameter '--retention-in-days 14' means logs are kept for 14 days.

3. Final Answer:

   Logs will be kept for 14 days -> Option C

4. Quick Check:

   Retention days = 14 [OK]

Hint: Retention days = value in --retention-in-days [OK]

Common Mistakes:

• Assuming default retention instead of set value
• Confusing retention period with log group creation
• Thinking logs are kept indefinitely by default

4. You run this command but get an error:

aws logs put-retention-policy --log-group-name NonExistentGroup --retention-in-days 30

What is the most likely cause?

medium

A. The log group 'NonExistentGroup' does not exist

B. Retention period 30 days is not allowed

C. You must specify a log stream name instead

D. The AWS CLI command is misspelled

Solution

1. Step 1: Check existence of log group

   The command requires the log group to exist before setting retention.

2. Step 2: Validate retention period and command syntax

   Retention of 30 days is valid and the command syntax is correct.

3. Final Answer:

   The log group 'NonExistentGroup' does not exist -> Option A

4. Quick Check:

   Retention needs existing log group [OK]

Hint: Retention policy requires existing log group [OK]

Common Mistakes:

• Assuming retention period is invalid
• Confusing log stream with log group
• Thinking command syntax is wrong

5. You want to organize logs from multiple microservices in CloudWatch Logs. Which approach best follows AWS best practices?

hard

A. Store all logs in a single log stream for simplicity

B. Create one log stream for all microservices in a single log group

C. Create one log group per instance and one log stream per microservice

D. Create one log group per microservice and multiple log streams per instance

Solution

1. Step 1: Understand log group and stream organization

   Best practice is to use log groups to separate different applications or microservices.

2. Step 2: Assign log streams per instance or source

   Within each log group, use multiple log streams to separate logs by instance or container.

3. Step 3: Evaluate options

   Create one log group per microservice and multiple log streams per instance matches best practice: one log group per microservice, multiple streams per instance.

4. Final Answer:

   Create one log group per microservice and multiple log streams per instance -> Option D

5. Quick Check:

   Group per microservice, streams per instance [OK]

Hint: Group by microservice, stream by instance [OK]

Common Mistakes:

• Mixing microservices in one log group
• Using one stream for all logs causing confusion
• Creating groups per instance instead of per service