Cluster Autoscaler concept in Kubernetes - Commands & Configuration

Introduction

Sometimes your Kubernetes cluster needs more or fewer worker nodes depending on the workload. Cluster Autoscaler automatically adjusts the number of nodes to match the demand, saving resources and cost.

When your app traffic suddenly increases and you need more nodes to handle the load.

When your cluster has idle nodes that are not running any pods and you want to save cost by removing them.

When you want your cluster to scale up or down automatically without manual intervention.

When you run batch jobs that require extra nodes temporarily.

When you want to maintain application availability by ensuring enough nodes are available.

Config File - cluster-autoscaler.yaml

cluster-autoscaler.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  labels:
    app: cluster-autoscaler
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      serviceAccountName: cluster-autoscaler
      containers:
      - name: cluster-autoscaler
        image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.26.0
        command:
          - ./cluster-autoscaler
          - --v=4
          - --stderrthreshold=info
          - --cloud-provider=aws
          - --skip-nodes-with-local-storage=false
          - --expander=least-waste
          - --nodes=1:10:example-node-group
        resources:
          limits:
            cpu: 100m
            memory: 300Mi
          requests:
            cpu: 100m
            memory: 300Mi
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        effect: NoSchedule
      - key: node.kubernetes.io/not-ready
        operator: Exists
        effect: NoExecute
      - key: node.kubernetes.io/unreachable
        operator: Exists
        effect: NoExecute

This file deploys the Cluster Autoscaler as a Deployment in the kube-system namespace.

The image specifies the autoscaler version.

The --nodes=1:10:example-node-group flag sets the minimum and maximum nodes for the node group named example-node-group.

Tolerations allow the autoscaler to run on control-plane nodes and handle node conditions.

Commands

This command creates the Cluster Autoscaler deployment in your Kubernetes cluster so it can start managing node scaling automatically.

Terminal

kubectl apply -f cluster-autoscaler.yaml

Expected Output

deployment.apps/cluster-autoscaler created

Check that the Cluster Autoscaler deployment is running and ready in the kube-system namespace.

Terminal

kubectl get deployment cluster-autoscaler -n kube-system

Expected Output

NAME                READY   UP-TO-DATE   AVAILABLE   AGE

cluster-autoscaler  1/1     1            1           30s

→

-n - Specifies the namespace where the deployment is located

View the logs of the Cluster Autoscaler to see its activity and confirm it is monitoring the cluster nodes.

Terminal

kubectl logs -f deployment/cluster-autoscaler -n kube-system

Expected Output

I0427 12:00:00.000000       1 cluster_autoscaler.go:123] Starting main loop

I0427 12:00:05.000000       1 scale_up.go:456] Scale-up: adding 1 node(s) to example-node-group

I0427 12:00:10.000000       1 scale_down.go:789] Scale-down: removing 1 node(s) from example-node-group

→

-f - Follows the log output in real time

→

-n - Specifies the namespace of the deployment

Key Concept

If you remember nothing else from this pattern, remember: Cluster Autoscaler automatically adds or removes nodes to match your workload demand, saving resources and keeping your apps running smoothly.

Common Mistakes

Not setting the correct node group name in the --nodes flag

The autoscaler won't know which nodes to scale, so it won't add or remove any nodes.

Use the exact node group name as configured in your cloud provider or cluster setup in the --nodes flag.

Not giving the autoscaler permission to run in the kube-system namespace

The deployment will fail or the autoscaler won't start because it lacks required permissions.

Create and assign the proper service account and roles for the autoscaler in kube-system.

Summary

Deploy the Cluster Autoscaler with a configuration specifying min and max nodes for your node group.

Verify the deployment is running and ready in the kube-system namespace.

Check autoscaler logs to monitor scaling actions and ensure it is working correctly.

Practice

1. What is the primary purpose of the Kubernetes Cluster Autoscaler?

easy

A. Manage pod deployments and replicas

B. Automatically adjust the number of nodes in a cluster based on workload

C. Monitor network traffic between nodes

D. Schedule pods to specific nodes manually

Solution

1. Step 1: Understand Cluster Autoscaler role

   The Cluster Autoscaler watches the cluster workload and adjusts node count automatically.

2. Step 2: Compare with other Kubernetes functions

   Pod management and scheduling are handled by other components, not autoscaler.

3. Final Answer:

   Automatically adjust the number of nodes in a cluster based on workload -> Option B

4. Quick Check:

   Cluster Autoscaler = Adjust nodes automatically [OK]

Hint: Autoscaler changes nodes count, not pods or network [OK]

Common Mistakes:

• Confusing autoscaler with pod scheduler
• Thinking it manages network or pods directly
• Assuming manual node scaling only

2. Which of the following is the correct way to specify the minimum and maximum size of a node group in Cluster Autoscaler configuration?

easy

A. --nodes=5:10:node-group-name

B. --min-nodes=5 --max-nodes=10 --node-group=node-group-name

C. --node-group=node-group-name --nodes=5-10

D. --scale=5,10,node-group-name

Solution

1. Step 1: Recall Cluster Autoscaler node group syntax

   The correct syntax uses --nodes=min:max:node-group-name format.

2. Step 2: Check other options for syntax errors

   Options A, B, and C do not match the documented syntax for Cluster Autoscaler.

3. Final Answer:

   --nodes=5:10:node-group-name -> Option A

4. Quick Check:

   Min:max:group = --nodes option [OK]

Hint: Look for min:max:group format in --nodes option [OK]

Common Mistakes:

• Using separate flags for min and max nodes
• Using commas or dashes incorrectly
• Misplacing node group name

3. Given the following Cluster Autoscaler configuration snippet:

--nodes=2:5:group-a
--nodes=1:3:group-b

What is the maximum number of nodes the autoscaler can add to the cluster?

medium

A. 5 nodes total

B. Cannot determine from this configuration

C. 3 nodes total

D. 8 nodes total

Solution

1. Step 1: Identify max nodes per group

   Group A max nodes = 5, Group B max nodes = 3.

2. Step 2: Sum max nodes for total max

   Total max nodes = 5 + 3 = 8 nodes.

3. Final Answer:

   8 nodes total -> Option D

4. Quick Check:

   Sum max nodes = 8 [OK]

Hint: Add max nodes from all groups for total max [OK]

Common Mistakes:

• Adding min nodes instead of max
• Counting only one group
• Assuming max is fixed cluster size

4. You configured Cluster Autoscaler with:

--nodes=1:4:group-x

But the cluster never scales down nodes even when they are idle. What could be a likely cause?

medium

A. Scale down is disabled or scale down delay is too long

B. Cluster Autoscaler only scales up, not down

C. Minimum nodes is set to 1, so it never removes nodes below that

D. Node group name is incorrect

Solution

1. Step 1: Consider scale-down configuration

   Scale down might be disabled or the scale-down delays (like --scale-down-delay-after-add) are set too long, causing no scale-down even for idle nodes.

2. Step 2: Distinguish from other options

   Min nodes=1 allows scaling down to 1 but not below; incorrect node group would affect scale-up too.

3. Final Answer:

   Scale down is disabled or scale down delay is too long -> Option A

4. Quick Check:

   Disabled scale-down or long delay = no scaling down [OK]

Hint: Check if scale-down is disabled or delay too high [OK]

Common Mistakes:

• Confusing with minimum nodes limit
• Assuming autoscaler can't scale down
• Blaming node group name without checking config

5. You want to optimize costs by configuring Cluster Autoscaler to remove unused nodes quickly but avoid removing nodes that might be needed soon. Which configuration approach is best?

hard

A. Disable scale-down and manually remove nodes

B. Set max nodes equal to min nodes to prevent scaling

C. Set a low scale-down delay and enable scale-down of empty nodes

D. Set a high scale-down delay and disable scale-down of empty nodes

Solution

1. Step 1: Understand scale-down delay and empty node removal

   Low scale-down delay removes unused nodes faster; enabling empty node scale-down helps remove idle nodes.

2. Step 2: Avoid preventing scaling or disabling scale-down

   Setting max = min or disabling scale-down prevents autoscaler from saving costs.

3. Final Answer:

   Set a low scale-down delay and enable scale-down of empty nodes -> Option C

4. Quick Check:

   Fast scale-down + empty node removal = cost optimization [OK]

Hint: Lower delay + empty node scale-down = save costs fast [OK]

Common Mistakes:

• Setting max = min disables scaling
• Disabling scale-down wastes resources
• Using high delay delays cost savings