How to Use AWS SQS with Lambda for Event-Driven Processing

You can use AWS Lambda to process messages from an SQS queue by configuring the queue as an event source for the Lambda function. This setup triggers the Lambda automatically when new messages arrive, allowing serverless, scalable processing without polling.

Syntax

To connect an SQS queue to a Lambda function, you create an event source mapping. This mapping tells Lambda which queue to listen to and how to handle messages.

• Queue ARN: The Amazon Resource Name of the SQS queue.
• Function ARN: The Lambda function's ARN to invoke.
• Batch size: Number of messages Lambda receives per invocation.
• Enabled: Whether the event source mapping is active.

bash

aws lambda create-event-source-mapping \
  --function-name MyLambdaFunction \
  --batch-size 10 \
  --event-source-arn arn:aws:sqs:region:account-id:MyQueue \
  --enabled

Example

This example shows a simple AWS Lambda function in Python that processes messages from an SQS queue. The Lambda reads each message and prints its content.

python

import json

def lambda_handler(event, context):
    for record in event['Records']:
        message_body = record['body']
        print(f"Received message: {message_body}")
    return {'statusCode': 200, 'body': json.dumps('Messages processed')}

Output

Received message: Hello from SQS
Received message: Another message

Common Pitfalls

• Missing permissions: Lambda needs permission to read from the SQS queue. Attach an IAM role with sqs:ReceiveMessage, sqs:DeleteMessage, and sqs:GetQueueAttributes actions.
• Incorrect event source ARN: Use the full ARN of the SQS queue, not just the name.
• Batch size too large: Large batch sizes can increase processing time and Lambda cost.
• Not handling message deletion: Lambda automatically deletes messages only on successful processing; errors cause retries.

json

## Wrong: Missing permissions
# Lambda role without SQS permissions causes failures

## Right: Add this IAM policy snippet to Lambda role
{
  "Effect": "Allow",
  "Action": ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes"],
  "Resource": "arn:aws:sqs:region:account-id:MyQueue"
}

Quick Reference

Concept	Description
Event Source Mapping	Connects SQS queue to Lambda for automatic triggers
Batch Size	Number of messages Lambda processes per invocation (1-10)
IAM Role	Lambda needs permissions to read and delete messages from SQS
Message Deletion	Messages are deleted only after successful Lambda processing
Error Handling	Failed messages are retried or sent to a dead-letter queue

Key Takeaways

Configure an event source mapping to link your SQS queue to your Lambda function.

Ensure your Lambda's IAM role has permissions to receive and delete messages from SQS.

Set a reasonable batch size to balance processing efficiency and cost.

Lambda automatically deletes messages only after successful processing to avoid data loss.

Use dead-letter queues to handle messages that fail processing repeatedly.