CloudWatch Logs in AWS - Time & Space Complexity
When working with CloudWatch Logs, it's important to understand how the time to process logs grows as you add more log events.
We want to know how the number of API calls changes as the amount of log data increases.
Analyze the time complexity of the following operation sequence.
// Put multiple log events into a log stream
aws logs put-log-events \
--log-group-name MyLogGroup \
--log-stream-name MyLogStream \
--log-events '[{"timestamp": 1609459200000, "message": "Log message 1"}, {"timestamp": 1609459201000, "message": "Log message 2"}]'
// Retrieve log events from the log stream
aws logs get-log-events \
--log-group-name MyLogGroup \
--log-stream-name MyLogStream
This sequence sends batches of log messages to CloudWatch Logs and then retrieves them from the log stream.
Identify the API calls, resource provisioning, data transfers that repeat.
- Primary operation: The
put-log-eventsAPI call to send log batches. - How many times: Once per batch of log events sent.
- Secondary operation: The
get-log-eventsAPI call to read logs. - How many times: Once per retrieval request, which may paginate if logs are large.
As you increase the number of log events, the number of put-log-events calls grows roughly in proportion to how many batches you need.
| Input Size (n log events) | Approx. API Calls |
|---|---|
| 10 | 1 (one batch) |
| 1000 | 1 (one batch) |
| 100000 | About 100 batches |
Pattern observation: The number of API calls grows linearly with the number of log events divided by batch size.
Time Complexity: O(n)
This means the time to send or retrieve logs grows directly in proportion to the number of log events.
[X] Wrong: "Sending more logs will always take the same time because it's just one API call."
[OK] Correct: CloudWatch limits batch sizes, so large numbers of logs require multiple API calls, increasing total time.
Understanding how log ingestion scales helps you design systems that handle large volumes efficiently and predict costs and delays.
"What if we changed the batch size limit to a smaller number? How would the time complexity change?"