CloudWatch Logs in AWS - Commands & Configuration

Introduction

CloudWatch Logs helps you collect and store logs from your applications and servers. It solves the problem of finding and analyzing errors or events by keeping all logs in one place.

When you want to see error messages from your web server to fix bugs quickly.

When you need to monitor application behavior over time to improve performance.

When you want to keep track of security events like login attempts.

When you want to create alerts based on specific log messages.

When you want to store logs safely for future audits or troubleshooting.

Config File - cloudwatch-log-group.json

cloudwatch-log-group.json

{
  "Resources": {
    "MyLogGroup": {
      "Type": "AWS::Logs::LogGroup",
      "Properties": {
        "LogGroupName": "my-app-logs",
        "RetentionInDays": 14
      }
    }
  }
}

This JSON file defines a CloudWatch Log Group named my-app-logs. The RetentionInDays key sets logs to be kept for 14 days before automatic deletion. This helps manage storage and costs.

Commands

This command creates a new CloudWatch Log Group named 'my-app-logs' to store your logs.

Terminal

aws logs create-log-group --log-group-name my-app-logs

Expected Output

No output (command runs silently)

→

--log-group-name - Specifies the name of the log group to create

This command sets the log retention period to 14 days, so logs older than that are deleted automatically.

Terminal

aws logs put-retention-policy --log-group-name my-app-logs --retention-in-days 14

Expected Output

No output (command runs silently)

→

--retention-in-days - Sets how many days to keep the logs

This command lists log groups starting with 'my-app-logs' to verify the log group was created.

Terminal

aws logs describe-log-groups --log-group-name-prefix my-app-logs

Expected Output

{"logGroups":[{"logGroupName":"my-app-logs","creationTime":1686000000000,"retentionInDays":14}]}

→

--log-group-name-prefix - Filters log groups by name prefix

This command sends a log event with a timestamp and message to the log stream 'my-stream' inside the 'my-app-logs' group.

Terminal

aws logs put-log-events --log-group-name my-app-logs --log-stream-name my-stream --log-events timestamp=1686000000000,message="Application started"

Expected Output

{"nextSequenceToken":"4953670123456789012345678901234567890123456789012345678901234567890"}

→

--log-stream-name - Specifies the log stream to send logs to

→

--log-events - Contains the log messages with timestamps

This command retrieves the log events from the 'my-stream' log stream to check the logs you sent.

Terminal

aws logs get-log-events --log-group-name my-app-logs --log-stream-name my-stream

Expected Output

{"events":[{"timestamp":1686000000000,"message":"Application started","ingestionTime":1686000001000}]}

→

--log-stream-name - Specifies which log stream to read from

Key Concept

If you remember nothing else from this pattern, remember: CloudWatch Logs groups and streams organize your logs so you can find and analyze them easily.

Common Mistakes

Trying to send logs to a log stream that does not exist.

CloudWatch Logs requires the log stream to exist before sending logs, otherwise the command fails.

Create the log stream first using 'aws logs create-log-stream' before sending logs.

Not setting a retention policy, causing logs to accumulate indefinitely.

Without retention, logs keep growing and can increase costs and storage usage.

Always set a retention period with 'put-retention-policy' to manage log storage.

Using incorrect timestamp format when sending log events.

Timestamps must be in milliseconds since epoch; wrong format causes errors or wrong log order.

Use Unix time in milliseconds for the timestamp value.

Summary

Create a CloudWatch Log Group to organize your logs.

Set a retention policy to automatically delete old logs and save costs.

Send log events to a log stream inside the log group.

Retrieve log events to verify and analyze your logs.