Lambda execution model in AWS - Deep Dive

Overview - Lambda execution model

What is it?

The Lambda execution model describes how AWS Lambda runs your code in response to events. It manages the process of starting, running, and stopping your code without you needing to handle servers. When an event triggers your Lambda function, AWS creates an environment to run your code, executes it, and then cleans up or reuses the environment for future requests.

Why it matters

This model lets developers run code without managing servers, saving time and reducing complexity. Without it, you would need to set up and maintain servers to handle every request, which is costly and slow. Lambda's execution model makes applications scalable and efficient by automatically handling how and when your code runs.

Where it fits

Before learning this, you should understand basic cloud computing and event-driven programming. After this, you can explore advanced Lambda features like concurrency, cold starts, and integration with other AWS services.

Mental Model

Core Idea

AWS Lambda runs your code inside short-lived containers that start on demand, execute your function, and then pause or stop, automatically managing resources for you.

Think of it like...

It's like a food truck that sets up only when customers arrive, serves the meal quickly, and then packs up or stays ready for the next customer without needing a full restaurant.

┌───────────────┐        ┌───────────────┐        ┌───────────────┐
│ Event arrives │ ─────> │ Container     │ ─────> │ Code runs     │
│ triggers     │        │ starts        │        │ executes      │
└───────────────┘        └───────────────┘        └───────────────┘
         │                      │                        │
         │                      ▼                        ▼
         │              ┌───────────────┐        ┌───────────────┐
         │              │ Container     │        │ Container     │
         └─────────────>│ pauses or     │ <───── │ reused for    │
                        │ stops         │        │ next event    │
                        └───────────────┘        └───────────────┘

Build-Up - 7 Steps

1

Foundation

What is AWS Lambda

Concept: Introduction to AWS Lambda as a serverless compute service.

AWS Lambda lets you run code without managing servers. You write a function, upload it, and Lambda runs it when triggered by events like file uploads or web requests.

Result

You can run code automatically in response to events without worrying about servers.

Understanding Lambda as a service that abstracts servers is key to grasping its execution model.

2

Foundation

Event-driven invocation basics

Concept: How Lambda functions are triggered by events.

Events from AWS services or external sources trigger Lambda functions. Each event causes Lambda to start running your code to handle that event.

Result

Your code runs only when needed, triggered by specific events.

Knowing that Lambda runs code only on demand helps understand resource efficiency.

3

Intermediate

Lambda container lifecycle

🤔Before reading on: do you think Lambda creates a new container for every event or reuses containers? Commit to your answer.

Concept: Lambda runs code inside containers that can be reused for multiple events.

When Lambda receives an event, it creates a container (an isolated environment) to run your code. After running, the container may pause and be reused for future events to save time. If no events come, the container stops eventually.

Result

Reusing containers reduces startup delays and improves performance.

Understanding container reuse explains why some Lambda invocations are faster than others.

4

Intermediate

Cold start vs warm start

🤔Before reading on: do you think all Lambda invocations start instantly or some take longer? Commit to your answer.

Concept: Cold starts happen when Lambda creates a new container; warm starts reuse existing containers.

A cold start is when Lambda must set up a new container and initialize your code, causing a delay. Warm starts happen when Lambda reuses a container, making execution faster. Cold starts happen after inactivity or scaling up.

Result

Cold starts cause delays; warm starts are faster and more efficient.

Knowing cold and warm starts helps optimize Lambda performance and user experience.

5

Intermediate

Execution environment details

Concept: What the Lambda container includes and how it runs your code.

Each Lambda container has a runtime (like Node.js or Python), your code, and temporary storage. It provides CPU, memory, and network access. The environment is isolated and secure, and it resets between cold starts.

Result

Your code runs in a controlled, secure environment with defined resources.

Understanding the execution environment helps in writing efficient and secure Lambda functions.

6

Advanced

Concurrency and scaling behavior

🤔Before reading on: do you think Lambda limits how many functions run at once or scales infinitely? Commit to your answer.

Concept: Lambda automatically scales by running multiple containers concurrently for multiple events.

When many events arrive, Lambda runs multiple containers in parallel, each handling one event. AWS limits concurrency per account and region, but Lambda scales automatically within those limits. Containers are created as needed, causing cold starts during scaling.

Result

Your application can handle many requests at once without manual scaling.

Knowing how Lambda scales helps design applications that handle variable workloads smoothly.

7

Expert

Container reuse pitfalls and optimization

🤔Before reading on: do you think data stored in one Lambda invocation is always safe for the next? Commit to your answer.

Concept: Containers may be reused, so data in memory can persist unexpectedly between invocations.

Because Lambda reuses containers, variables or connections opened in one invocation might still exist in the next. This can improve performance but cause bugs if your code assumes a fresh start every time. Proper initialization and cleanup are essential.

Result

Optimized Lambda functions run faster but require careful state management.

Understanding container reuse nuances prevents subtle bugs and improves Lambda efficiency.

Under the Hood

AWS Lambda runs your code inside lightweight containers managed by AWS. When an event triggers your function, Lambda either reuses an existing container or creates a new one (cold start). The container includes the runtime, your code, and allocated resources. After execution, the container pauses and may be reused for future events. AWS manages container lifecycle, scaling, and resource allocation automatically.

Why designed this way?

This design balances fast response times and resource efficiency. Creating a new container for every event would be slow and costly. Reusing containers reduces latency and cost. AWS chose containers for isolation and portability, enabling multi-language support and secure execution.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Event arrives │──────▶│ Check for     │──────▶│ Container     │
│ triggers     │       │ existing      │       │ reuse or new  │
└───────────────┘       │ container     │       └───────────────┘
                        └───────────────┘               │
                                │                        ▼
                                │               ┌───────────────┐
                                │               │ Run code in   │
                                │               │ container     │
                                │               └───────────────┘
                                │                        │
                                ▼                        ▼
                      ┌─────────────────┐       ┌───────────────┐
                      │ Container pauses│◀──────│ Return result │
                      │ or stops        │       └───────────────┘
                      └─────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think Lambda containers are always new for each event? Commit to yes or no.

Common Belief: Lambda creates a brand-new container every time it runs a function.

Reality: Lambda often reuses containers for multiple events to improve performance.

Why it matters: Assuming new containers always leads to ignoring persistent state, causing bugs or missed optimization opportunities.

Quick: Do you think Lambda functions run continuously like a server? Commit to yes or no.

Common Belief: Lambda functions run all the time like traditional servers.

Reality: Lambda functions run only when triggered by events and stop afterward.

Why it matters: Thinking Lambda runs continuously can cause confusion about billing and resource usage.

Quick: Do you think Lambda scales infinitely without limits? Commit to yes or no.

Common Belief: Lambda can handle unlimited concurrent executions without restrictions.

Reality: Lambda has concurrency limits per account and region, which can be increased but are not infinite.

Why it matters: Ignoring concurrency limits can cause unexpected throttling and failures in high-load applications.

Quick: Do you think data stored in one Lambda invocation is always cleared before the next? Commit to yes or no.

Common Belief: Each Lambda invocation starts with a completely clean environment.

Reality: Data in memory can persist between invocations if the container is reused.

Why it matters: Assuming a clean environment can cause bugs when code relies on fresh state every time.

Expert Zone

1

Lambda container reuse can cause subtle bugs if global variables or connections are not managed carefully.

2

Cold start latency varies by runtime, package size, and VPC configuration, affecting user experience.

3

Provisioned concurrency can be used to keep containers warm and reduce cold start delays for critical workloads.

When NOT to use

Lambda is not suitable for long-running processes or applications requiring persistent local storage. For such cases, use EC2 instances, ECS containers, or AWS Fargate instead.

Production Patterns

In production, Lambda functions are often combined with API Gateway for web APIs, triggered by S3 or DynamoDB events for data processing, and use environment variables and layers for configuration and shared code.

Connections

Containerization (Docker)

Lambda containers are lightweight, ephemeral containers similar to Docker containers.

Understanding Docker helps grasp how Lambda isolates and runs code securely and efficiently.

Event-driven architecture

Lambda's execution model is a practical implementation of event-driven design.

Knowing event-driven principles clarifies why Lambda runs code only on demand and scales automatically.

Operating system process scheduling

Lambda's container lifecycle resembles how an OS schedules and manages processes.

Understanding OS scheduling helps appreciate Lambda's balance between resource use and responsiveness.

Common Pitfalls

#1 Assuming all Lambda invocations have fresh state and opening new database connections every time.

Wrong approach:

def handler(event, context):
conn = open_database_connection()
# use conn
conn.close()

Correct approach:

conn = open_database_connection()
def handler(event, context):
# reuse conn
pass

Root cause:

Not realizing that containers can be reused, so opening connections outside the handler improves performance.

#2 Ignoring cold start delays and expecting instant response for all invocations.

Wrong approach:

def handler(event, context):
# heavy initialization inside handler
setup_heavy_resources()
return process(event)

Correct approach:

def initialize():
# heavy setup done once
pass
initialize()
def handler(event, context):
return process(event)

Root cause:

Placing heavy setup inside the handler causes cold start delays every time.

#3 Deploying large Lambda packages without considering startup time.

Wrong approach:

Uploading a Lambda function with hundreds of MBs of code and dependencies.

Correct approach:

Use Lambda layers and minimize package size to reduce cold start latency.

Root cause:

Not optimizing package size increases cold start time and costs.

Key Takeaways

AWS Lambda runs your code inside containers that start on demand and may be reused for multiple events.

Cold starts happen when a new container is created, causing delays; warm starts reuse containers and run faster.

Lambda automatically scales by running multiple containers concurrently but has concurrency limits to consider.

Understanding container reuse is crucial to avoid bugs and optimize performance in Lambda functions.

Lambda's execution model enables serverless, event-driven applications without managing servers or infrastructure.