Why Cluster upgrade strategies in Kubernetes? - Purpose & Use Cases

The Big Idea

What if you could update your entire system without your users ever noticing?

The Scenario

Imagine you have a group of servers running your applications, and you need to update them all to a new version. You try to do this by turning off each server one by one, updating it, and then turning it back on manually.

The Problem

This manual way is slow and risky. If you update all servers at once, your whole system might stop working. If you update them one by one without a plan, users might face downtime or errors. It's easy to make mistakes and hard to fix problems quickly.

The Solution

Cluster upgrade strategies help you update your servers smoothly and safely. They guide you to update parts of your system step-by-step, checking that everything works before moving on. This way, your applications stay available and users don't notice any interruptions.

Before vs After

✗ Before

shutdown server1
update server1
start server1
shutdown server2
update server2
start server2

✓ After

kubectl drain node1
kubeadm upgrade node1
kubectl uncordon node1
kubectl drain node2
kubeadm upgrade node2
kubectl uncordon node2

What It Enables

It enables seamless updates with zero downtime, keeping your services reliable and users happy.

Real Life Example

A company running an online store uses cluster upgrade strategies to update their servers without stopping customers from shopping, even during big software changes.

Key Takeaways

Manual updates can cause downtime and errors.

Cluster upgrade strategies automate safe, step-by-step updates.

This keeps applications running smoothly during upgrades.

Practice

1. What is the recommended order when upgrading a Kubernetes cluster?

easy

A. Upgrade all nodes simultaneously

B. Upgrade worker nodes first, then control plane nodes

C. Upgrade control plane nodes first, then worker nodes

D. Upgrade only the worker nodes

Solution

1. Step 1: Understand the role of control plane nodes

   Control plane nodes manage the cluster state and API server, so they must be stable first.

2. Step 2: Upgrade worker nodes after control plane

   Worker nodes run workloads and depend on the control plane, so upgrade them after control plane nodes.

3. Final Answer:

   Upgrade control plane nodes first, then worker nodes -> Option C

4. Quick Check:

   Control plane first, workers second = A [OK]

Hint: Always upgrade control plane nodes before worker nodes [OK]

Common Mistakes:

• Upgrading worker nodes before control plane
• Upgrading all nodes at once causing downtime
• Skipping control plane upgrade

2. Which command correctly drains a node before upgrading it?

easy

A. kubectl drain --ignore-daemonsets --delete-local-data

B. kubectl upgrade node

C. kubectl delete node

D. kubectl cordon --force

Solution

1. Step 1: Identify the correct drain command syntax

   The command to safely evict pods is 'kubectl drain' with flags to ignore daemonsets and delete local data.

2. Step 2: Verify other options are incorrect

   Upgrade and delete commands do not drain nodes; cordon only marks unschedulable but does not evict pods.

3. Final Answer:

   kubectl drain <node-name> --ignore-daemonsets --delete-local-data -> Option A

4. Quick Check:

   Drain command with correct flags = A [OK]

Hint: Use 'kubectl drain' with flags to safely evict pods [OK]

Common Mistakes:

• Using 'kubectl cordon' instead of 'drain'
• Deleting nodes instead of draining
• Missing flags causing pod eviction failure

3. Given this upgrade sequence, what is the expected cluster state?

1. Drain node1
2. Upgrade node1
3. Uncordon node1
4. Repeat for node2 and node3

medium

A. Control plane nodes are upgraded last

B. Cluster remains available with minimal downtime

C. Pods are deleted permanently during upgrade

D. Cluster goes down during node upgrades

Solution

1. Step 1: Analyze the upgrade steps

   Each node is drained to safely evict pods, upgraded, then uncordoned to resume scheduling.

2. Step 2: Understand impact on cluster availability

   Upgrading nodes one by one with draining keeps workloads running on other nodes, minimizing downtime.

3. Final Answer:

   Cluster remains available with minimal downtime -> Option B

4. Quick Check:

   Draining and upgrading nodes one by one = D [OK]

Hint: Upgrade nodes one at a time with drain/un-cordon for uptime [OK]

Common Mistakes:

• Assuming cluster goes down during upgrades
• Not draining nodes causing pod failures
• Upgrading all nodes simultaneously

4. You ran kubectl drain node1 but pods did not evict. What is the likely cause?

medium

A. DaemonSet pods are blocking eviction

B. Node is already uncordoned

C. Control plane node cannot be drained

D. Pods have no local storage

Solution

1. Step 1: Understand drain behavior with DaemonSets

   By default, drain blocks if DaemonSet pods are running unless --ignore-daemonsets is used.

2. Step 2: Check other options for correctness

   Uncordon status does not block eviction; control plane nodes can be drained; pods without local storage do not block drain.

3. Final Answer:

   DaemonSet pods are blocking eviction -> Option A

4. Quick Check:

   DaemonSet pods block drain without flag = C [OK]

Hint: Use --ignore-daemonsets flag to drain nodes with DaemonSet pods [OK]

Common Mistakes:

• Not using --ignore-daemonsets flag
• Confusing cordon with drain
• Assuming control plane nodes cannot be drained

5. You want to upgrade a large Kubernetes cluster with minimal downtime. Which strategy is best?

hard

A. Upgrade all control plane nodes simultaneously, then all workers simultaneously

B. Skip draining and upgrade nodes in random order

C. Drain all nodes at once, upgrade, then uncordon all nodes

D. Use cloud provider upgrade tools to upgrade control plane, then drain and upgrade workers one by one

Solution

1. Step 1: Consider cloud provider tools for control plane upgrade

   Cloud tools often automate safe control plane upgrades reducing manual errors.

2. Step 2: Upgrade worker nodes one by one with drain/un-cordon

   This approach avoids downtime by keeping workloads running on other nodes during upgrade.

3. Step 3: Evaluate other options for risks

   Upgrading all nodes simultaneously or skipping drain risks downtime and pod failures.

4. Final Answer:

   Use cloud provider upgrade tools to upgrade control plane, then drain and upgrade workers one by one -> Option D

5. Quick Check:

   Cloud tools + sequential worker upgrade = B [OK]

Hint: Use cloud tools and upgrade workers one at a time with drain [OK]

Common Mistakes:

• Upgrading all nodes simultaneously causing downtime
• Skipping drain causing pod disruption
• Ignoring cloud provider upgrade features