AwsHow-ToBeginner · 4 min read

How to Set Up Log Subscription Filter in AWS

To set up a log subscription filter in AWS, use the aws logs put-subscription-filter command or AWS Console to connect a CloudWatch Logs group to a destination like a Lambda function or Kinesis stream. Define a filterPattern to specify which log events to send, and provide the destinationArn for where logs should be delivered.

📐

Syntax

The main command to create a log subscription filter is aws logs put-subscription-filter. It requires these key parts:

log-group-name: The name of the CloudWatch Logs group to subscribe from.
filter-name: A name you choose for the subscription filter.
filter-pattern: A pattern to select which log events to send.
destination-arn: The ARN of the destination (Lambda, Kinesis, or Firehose) to receive the logs.

bash

aws logs put-subscription-filter \
  --log-group-name <log-group-name> \
  --filter-name <filter-name> \
  --filter-pattern '<filter-pattern>' \
  --destination-arn <destination-arn>

💻

Example

This example creates a subscription filter that sends all error logs from the /aws/lambda/my-function log group to a Lambda function for processing.

bash

aws logs put-subscription-filter \
  --log-group-name /aws/lambda/my-function \
  --filter-name ErrorFilter \
  --filter-pattern 'ERROR' \
  --destination-arn arn:aws:lambda:us-east-1:123456789012:function:ProcessLogs

Output

Subscription filter 'ErrorFilter' created for log group '/aws/lambda/my-function'.

⚠️

Common Pitfalls

Incorrect destination ARN: Make sure the destination ARN is valid and the destination service has the right permissions.
Missing permissions: The CloudWatch Logs service needs permission to invoke the destination (e.g., Lambda). Attach the correct resource policy.
Filter pattern errors: Using an invalid filter pattern will cause the subscription to fail. Test patterns carefully.
One subscription per log group: Each log group can have only one subscription filter. To change it, delete the old one first.

bash

aws logs delete-subscription-filter --log-group-name /aws/lambda/my-function --filter-name ErrorFilter

aws logs put-subscription-filter \
  --log-group-name /aws/lambda/my-function \
  --filter-name NewFilter \
  --filter-pattern 'WARN' \
  --destination-arn arn:aws:kinesis:us-east-1:123456789012:stream/MyStream

Output

Subscription filter 'ErrorFilter' deleted. Subscription filter 'NewFilter' created for log group '/aws/lambda/my-function'.

📊

Quick Reference

Parameter	Description
`log-group-name`	Name of the CloudWatch Logs group to subscribe to.
`filter-name`	Name for the subscription filter.
`filter-pattern`	Pattern to select log events (e.g., 'ERROR', '{ $.status = 404 }').
`destination-arn`	ARN of the destination (Lambda, Kinesis, Firehose).

Parameter	Description
log-group-name	Name of the CloudWatch Logs group to subscribe to.
filter-name	Name for the subscription filter.
filter-pattern	Pattern to select log events (e.g., 'ERROR', '{ $.status = 404 }').
destination-arn	ARN of the destination (Lambda, Kinesis, Firehose).

✅

Key Takeaways

Use aws logs put-subscription-filter to connect a log group to a destination with a filter pattern.

Each log group can have only one subscription filter at a time.

Ensure the destination has proper permissions to receive logs from CloudWatch.

Test filter patterns carefully to capture the right log events.

Delete existing subscription filters before creating new ones on the same log group.