EKS cluster creation in AWS - Deep Dive

Overview - EKS cluster creation

What is it?

EKS cluster creation is the process of setting up a managed Kubernetes environment on AWS. It involves provisioning the control plane and worker nodes that run containerized applications. This service simplifies running Kubernetes by handling the complex parts like scaling and patching. Beginners can deploy and manage applications without managing the underlying infrastructure.

Why it matters

Without EKS, running Kubernetes requires managing servers, networking, and updates manually, which is complex and error-prone. EKS solves this by automating cluster management, letting teams focus on building applications. This reduces downtime, improves security, and speeds up delivery. Without it, many companies would struggle to run reliable containerized apps at scale.

Where it fits

Before learning EKS cluster creation, you should understand basic cloud concepts, AWS core services like EC2 and IAM, and container basics. After mastering EKS cluster creation, you can learn advanced Kubernetes operations, CI/CD pipelines for containers, and multi-cluster management.

Mental Model

Core Idea

EKS cluster creation is like ordering a fully managed container playground where AWS builds and runs the control center, while you add and manage the worker machines.

Think of it like...

Imagine renting a playground where the owner builds and maintains the main play structures (control plane), and you bring your own toys and friends (worker nodes) to play safely and easily.

┌─────────────────────────────┐
│       AWS Control Plane      │
│  (Managed Kubernetes Master)│
└─────────────┬───────────────┘
              │
      ┌───────┴────────┐
      │ Worker Nodes    │
      │ (EC2 Instances) │
      └─────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding Kubernetes Basics

Concept: Learn what Kubernetes is and why it manages containers.

Kubernetes is a system that helps run many containers together. It organizes containers into groups called pods and manages their lifecycle. It makes sure your apps keep running even if some parts fail.

Result

You know the role of Kubernetes and why it needs a control plane and worker nodes.

Understanding Kubernetes basics is essential because EKS is a managed Kubernetes service, so knowing what it manages helps you use it effectively.

2

Foundation

AWS Core Services for EKS

Concept: Learn the AWS services that support EKS cluster creation.

EKS uses EC2 for worker nodes, IAM for permissions, VPC for networking, and CloudWatch for logging. These services work together to run and secure your Kubernetes cluster.

Result

You can identify the AWS building blocks behind an EKS cluster.

Knowing these services helps you understand what EKS manages for you and what you must configure.

3

Intermediate

Creating the EKS Control Plane

🤔Before reading on: do you think you must manage the Kubernetes master nodes yourself or does AWS handle them? Commit to your answer.

Concept: AWS manages the Kubernetes control plane automatically when you create an EKS cluster.

When you create an EKS cluster, AWS provisions and runs the control plane in a highly available way. You don't see or manage these servers directly. You specify the cluster name, version, and networking settings.

Result

A running Kubernetes control plane managed by AWS is ready to accept worker nodes.

Knowing AWS manages the control plane frees you from complex setup and maintenance, letting you focus on your applications.

4

Intermediate

Provisioning Worker Nodes

🤔Before reading on: do you think worker nodes are automatically created with the cluster or do you need to add them separately? Commit to your answer.

Concept: Worker nodes are EC2 instances that you create and connect to the EKS control plane.

After the control plane is ready, you launch worker nodes using AWS managed node groups or self-managed nodes. These nodes join the cluster and run your container workloads. You configure their size, count, and permissions.

Result

Worker nodes join the cluster and start running pods scheduled by Kubernetes.

Understanding worker nodes as separate resources clarifies your responsibility in scaling and managing compute capacity.

5

Intermediate

Configuring Networking and Security

🤔Before reading on: do you think EKS automatically configures all networking and security or do you need to set up VPC and IAM roles? Commit to your answer.

Concept: You must configure networking (VPC, subnets) and security (IAM roles, security groups) for your EKS cluster and nodes.

EKS requires a VPC with subnets for the cluster and nodes to communicate. IAM roles define permissions for the control plane and nodes. Security groups control network traffic. These settings ensure secure and reliable cluster operation.

Result

A secure and connected EKS cluster environment ready for workloads.

Knowing you configure networking and security helps prevent common setup errors and security risks.

6

Advanced

Using Managed Node Groups

🤔Before reading on: do you think managed node groups simplify worker node management or add complexity? Commit to your answer.

Concept: Managed node groups automate provisioning, updating, and scaling of worker nodes.

AWS provides managed node groups that handle lifecycle tasks like patching and replacing nodes. You define the node group size and instance types, and AWS manages the rest. This reduces operational overhead.

Result

Worker nodes are easier to maintain and update with less manual work.

Understanding managed node groups helps you reduce operational burden and improve cluster reliability.

7

Expert

Cluster Autoscaler and Scaling Strategies

🤔Before reading on: do you think Kubernetes automatically scales nodes or do you need extra setup? Commit to your answer.

Concept: Cluster Autoscaler dynamically adjusts worker node count based on workload demand.

The Cluster Autoscaler watches pod resource needs and adds or removes nodes automatically. You configure it with permissions and policies. This ensures efficient resource use and cost savings.

Result

Your EKS cluster scales worker nodes up or down automatically to match workload.

Knowing how autoscaling works lets you optimize costs and performance in production environments.

Under the Hood

EKS creates a Kubernetes control plane that runs in AWS-managed infrastructure across multiple availability zones for high availability. The control plane includes API servers, etcd database, and controllers. Worker nodes are EC2 instances that register with the control plane using secure certificates. Communication happens over the VPC network. IAM roles control permissions for API access and node operations. AWS continuously monitors and patches the control plane, while you manage worker nodes and workloads.

Why designed this way?

AWS designed EKS to offload the complex and error-prone control plane management from users, improving security and reliability. By separating control plane and worker nodes, AWS allows flexible scaling and customization. Alternatives like self-managed Kubernetes require users to handle all infrastructure, which is costly and risky. EKS balances control and convenience by managing the master nodes and letting users manage worker nodes.

┌─────────────────────────────┐
│       AWS Control Plane      │
│ ┌───────────────┐           │
│ │ API Server    │◄──────────┤
│ │ etcd Database │           │
│ │ Controllers   │           │
│ └───────────────┘           │
└─────────────┬───────────────┘
              │
      ┌───────┴────────┐
      │ Worker Nodes    │
      │ (EC2 Instances) │
      └───────┬────────┘
              │
      ┌───────┴────────┐
      │ VPC Networking │
      └────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think AWS manages both control plane and worker nodes automatically in EKS? Commit yes or no.

Common Belief: AWS manages everything in EKS, including worker nodes, so I don't need to configure or maintain them.

Reality: AWS manages only the control plane; you must create, configure, and maintain worker nodes yourself or use managed node groups.

Why it matters: Assuming AWS manages worker nodes can lead to missing critical updates, scaling issues, and security risks.

Quick: Do you think EKS clusters can run without a VPC? Commit yes or no.

Common Belief: EKS clusters can run without setting up a VPC because AWS handles networking automatically.

Reality: EKS requires a properly configured VPC with subnets for cluster networking; you must set this up or use AWS-provided templates.

Why it matters: Ignoring VPC setup causes cluster creation failures or unreachable nodes.

Quick: Do you think Kubernetes control plane nodes are visible and accessible to users in EKS? Commit yes or no.

Common Belief: I can access and manage the Kubernetes master nodes directly in EKS.

Reality: EKS control plane nodes are fully managed by AWS and not accessible to users; you interact via the Kubernetes API endpoint.

Why it matters: Expecting direct access leads to confusion and wasted effort trying to manage unavailable resources.

Quick: Do you think Cluster Autoscaler is enabled by default in EKS? Commit yes or no.

Common Belief: EKS automatically scales worker nodes up and down without extra configuration.

Reality: You must install and configure Cluster Autoscaler yourself; it is not enabled by default.

Why it matters: Assuming autoscaling is automatic can cause resource shortages or overspending.

Expert Zone

1

EKS control plane runs in a separate AWS account managed by AWS, isolating it from your resources for security.

2

Managed node groups use AWS Auto Scaling groups under the hood, but expose Kubernetes-friendly interfaces for easier management.

3

EKS supports multiple Kubernetes versions, but upgrading clusters requires careful planning to avoid downtime and compatibility issues.

When NOT to use

EKS is not ideal if you need full control over Kubernetes master nodes or want to run Kubernetes outside AWS. Alternatives include self-managed Kubernetes on EC2 or other cloud providers, or lightweight Kubernetes distributions for edge devices.

Production Patterns

In production, teams use managed node groups with autoscaling and spot instances for cost efficiency. They integrate EKS with CI/CD pipelines for automated deployments and use AWS IAM Roles for Service Accounts to securely grant permissions to pods.

Connections

Serverless Computing

complementary cloud service

Understanding EKS helps appreciate how container orchestration differs from serverless functions, offering more control but requiring more management.

Distributed Systems Theory

builds-on

EKS embodies distributed system principles like consensus, fault tolerance, and scaling, so knowing these theories deepens understanding of cluster behavior.

Supply Chain Management

analogous process

Just as supply chains coordinate many moving parts to deliver products efficiently, EKS coordinates many components to deliver applications reliably.

Common Pitfalls

#1 Skipping VPC configuration or using default VPC without proper subnet setup.

Wrong approach:

eksctl create cluster --name my-cluster

Correct approach:

eksctl create cluster --name my-cluster --vpc-private-subnets=subnet-12345,subnet-67890

Root cause:

Assuming EKS can create a fully functional network automatically without specifying subnets.

#2 Not assigning correct IAM roles to worker nodes causing permission errors.

Wrong approach:

Creating worker nodes without attaching the AmazonEKSWorkerNodePolicy IAM role.

Correct approach:

Attach AmazonEKSWorkerNodePolicy, AmazonEC2ContainerRegistryReadOnly, and AmazonEKS_CNI_Policy roles to worker nodes.

Root cause:

Misunderstanding the need for specific permissions for nodes to communicate with the control plane and AWS services.

#3 Assuming cluster autoscaling works out of the box without setup.

Wrong approach:

Deploying workloads expecting nodes to scale automatically without installing Cluster Autoscaler.

Correct approach:

Install and configure Cluster Autoscaler with proper IAM permissions and node group tags.

Root cause:

Believing autoscaling is a default Kubernetes or EKS feature without additional configuration.

Key Takeaways

EKS cluster creation sets up a managed Kubernetes control plane on AWS, simplifying container orchestration.

AWS manages the control plane, but you must create and manage worker nodes and configure networking and security.

Managed node groups and Cluster Autoscaler help automate node management and scaling, improving reliability and cost efficiency.

Proper VPC and IAM role configuration are critical to a working and secure EKS cluster.

Understanding EKS internals and common pitfalls helps avoid errors and build robust production systems.