Rolling update strategy in Kubernetes - Deep Dive

Overview - Rolling update strategy

What is it?

A rolling update strategy is a way to update applications running in Kubernetes without stopping the entire service. It updates pods one by one or in small batches, so the app stays available during the change. This method replaces old versions with new ones gradually, avoiding downtime. It is a common way to deploy new features or fixes safely.

Why it matters

Without rolling updates, updating an app could cause downtime, making users unable to access the service. This can hurt user experience and business reputation. Rolling updates solve this by keeping the app running while changes happen, ensuring continuous availability and smooth transitions. It also helps catch problems early by updating in small steps.

Where it fits

Before learning rolling updates, you should understand Kubernetes basics like pods, deployments, and services. After mastering rolling updates, you can explore advanced deployment strategies like blue-green or canary deployments, and learn how to automate updates with CI/CD pipelines.

Mental Model

Core Idea

A rolling update replaces old app versions with new ones gradually, keeping the service live and stable throughout the process.

Think of it like...

Imagine changing tires on a car while it is still driving slowly. You replace one tire at a time so the car never stops moving, ensuring a smooth ride without interruption.

Deployment
  ├─ Pod 1 (old version) ──> Pod 1 (new version)
  ├─ Pod 2 (old version) ──> Pod 2 (new version)
  ├─ Pod 3 (old version) ──> Pod 3 (new version)
  └─ ...

Each pod updates one by one, keeping the app running.

Build-Up - 7 Steps

1

Foundation

Understanding Kubernetes Deployments

Concept: Learn what a deployment is and how it manages pods in Kubernetes.

A deployment in Kubernetes is like a manager for your app's pods. It ensures the right number of pods are running and can update them when you change the app version. You create a deployment with a YAML file specifying the app image and number of replicas.

Result

You get a set of pods running your app, managed by the deployment controller.

Knowing deployments is key because rolling updates happen through them, not directly on pods.

2

Foundation

What is a Pod in Kubernetes?

Concept: Pods are the smallest units running your app containers in Kubernetes.

A pod holds one or more containers that share storage and network. When you update your app, pods are replaced with new ones running the updated containers. Pods are temporary and can be created or destroyed by deployments.

Result

You understand that pods are the actual running instances of your app.

Recognizing pods as replaceable units helps grasp how rolling updates swap old pods for new ones.

3

Intermediate

How Rolling Updates Work in Deployments

🤔Before reading on: do you think all pods update at once or one by one in a rolling update? Commit to your answer.

Concept: Rolling updates replace pods gradually to avoid downtime.

When you change the app version in a deployment, Kubernetes starts creating new pods with the new version. It deletes old pods only after new ones are ready. This happens one or a few pods at a time, controlled by update settings.

Result

Your app stays available during updates because some pods keep running the old version while others switch.

Understanding the gradual replacement prevents surprises about app availability during updates.

4

Intermediate

Controlling Update Speed with Max Surge and Max Unavailable

🤔Before reading on: do you think max surge allows more pods than desired or fewer? Commit to your answer.

Concept: Max surge and max unavailable control how many pods update simultaneously.

Max surge lets Kubernetes create extra pods temporarily during update, speeding up rollout. Max unavailable limits how many pods can be down at once, ensuring minimum availability. For example, max surge: 1 and max unavailable: 1 means one extra pod can be added and one pod can be offline during update.

Result

You can balance update speed and app availability by tuning these settings.

Knowing these controls helps customize updates for different app needs and risk levels.

5

Intermediate

Using kubectl to Perform Rolling Updates

Concept: Learn how to trigger and monitor rolling updates using kubectl commands.

To update an app, change the image version in the deployment YAML and apply it with kubectl apply -f deployment.yaml. You can watch progress with kubectl rollout status deployment/your-app. If something goes wrong, rollback with kubectl rollout undo deployment/your-app.

Result

You can safely update your app and handle failures without downtime.

Mastering kubectl commands gives you control and confidence during updates.

6

Advanced

Handling Update Failures and Rollbacks

🤔Before reading on: do you think Kubernetes automatically rolls back failed updates or requires manual action? Commit to your answer.

Concept: Kubernetes can detect failed updates and allows rollbacks to previous versions.

If new pods fail readiness checks or crash, Kubernetes pauses the update. You can check status and decide to rollback using kubectl rollout undo. This prevents broken versions from affecting users.

Result

Your app remains stable even if updates introduce bugs.

Knowing rollback mechanisms prevents prolonged outages and speeds recovery.

7

Expert

Surprises in Rolling Update Behavior and Edge Cases

🤔Before reading on: do you think changing labels triggers rolling updates automatically? Commit to your answer.

Concept: Certain changes do not trigger rolling updates, and update behavior can vary with readiness probes and pod disruption budgets.

Only changes to pod template fields trigger rolling updates. Changing labels or annotations outside pod spec won't update pods. Readiness probes affect when pods are considered ready, impacting update speed. Pod Disruption Budgets can limit how many pods update simultaneously, causing delays.

Result

You understand why some updates seem ignored and how to tune update behavior precisely.

Recognizing these subtleties avoids confusion and helps design reliable update processes.

Under the Hood

Kubernetes deployments use a controller loop that compares desired state (new app version) with current state (running pods). It creates new pods with updated specs and deletes old pods only after new ones pass readiness checks. This ensures continuous availability. The controller respects max surge and max unavailable settings to balance speed and stability.

Why designed this way?

Rolling updates were designed to avoid downtime during app changes, a critical need for production systems. Gradual replacement reduces risk by limiting exposure to faulty versions. Alternatives like stopping all pods at once cause outages, so rolling updates became the standard for safe deployments.

Desired State (New Version)
       ↓
┌─────────────────────────┐
│ Deployment Controller    │
│                         │
│  ┌───────────────┐      │
│  │ Create New Pod │─────▶│ New Pod Ready
│  └───────────────┘      │
│         │               │
│         ▼               │
│  ┌───────────────┐      │
│  │ Delete Old Pod│      │
│  └───────────────┘      │
└─────────────────────────┘
       ↑
Current State (Old Pods)

Myth Busters - 4 Common Misconceptions

Quick: Does changing any part of a deployment always trigger a rolling update? Commit yes or no.

Common Belief: Any change to a deployment, like labels or annotations, triggers a rolling update.

Reality: Only changes to the pod template spec trigger rolling updates. Changes outside pod spec do not replace pods.

Why it matters: Thinking all changes trigger updates can cause confusion when pods don't update as expected, leading to stale versions running.

Quick: Do you think rolling updates guarantee zero downtime in all cases? Commit yes or no.

Common Belief: Rolling updates always ensure zero downtime during deployment.

Reality: While rolling updates minimize downtime, misconfigured readiness probes or resource limits can cause brief outages.

Why it matters: Assuming zero downtime can lead to ignoring readiness checks or resource needs, causing unexpected service interruptions.

Quick: Does Kubernetes automatically rollback failed updates without user action? Commit yes or no.

Common Belief: Kubernetes automatically rolls back any failed rolling update.

Reality: Kubernetes pauses failed updates but requires manual rollback commands to revert changes.

Why it matters: Believing in automatic rollback may delay recovery and prolong outages if manual intervention is not done.

Quick: Can you speed up rolling updates by increasing max surge indefinitely? Commit yes or no.

Common Belief: Setting max surge to a very high number always speeds up updates without issues.

Reality: Too high max surge can overload cluster resources and cause instability.

Why it matters: Ignoring resource limits can cause crashes or slowdowns, negating update benefits.

Expert Zone

1

Rolling updates respect Pod Disruption Budgets, which can delay updates if minimum availability is not met.

2

Readiness probes timing directly affects how fast new pods are considered ready, impacting update speed.

3

Changing environment variables or config maps mounted as volumes may not trigger rolling updates unless pod spec changes.

When NOT to use

Rolling updates are not ideal for major database schema changes or stateful apps requiring coordinated migrations. Alternatives like blue-green deployments or canary releases provide safer options for complex changes.

Production Patterns

Teams use rolling updates combined with health checks and monitoring to automate safe rollouts. They often integrate with CI/CD pipelines and use feature flags to control exposure. Rollbacks are scripted for quick recovery.

Connections

Blue-Green Deployment

Alternative deployment strategy with zero downtime by switching traffic between two environments.

Understanding rolling updates helps appreciate blue-green as a more isolated but resource-heavy approach to safe deployments.

Continuous Integration/Continuous Deployment (CI/CD)

Rolling updates are often automated within CI/CD pipelines for seamless app delivery.

Knowing rolling updates clarifies how deployment automation maintains app availability during frequent releases.

Traffic Light Systems (Engineering)

Both use staged transitions to avoid sudden failures and maintain flow.

Recognizing staged updates as a control system helps understand the importance of gradual change in complex systems.

Common Pitfalls

#1 Updating deployment labels expecting pods to restart.

Wrong approach:

kubectl patch deployment myapp -p '{"metadata":{"labels":{"version":"v2"}}}'

Correct approach:

kubectl set image deployment/myapp myapp-container=myimage:v2

Root cause:

Misunderstanding that only pod template changes trigger rolling updates, not metadata label changes.

#2 Setting maxUnavailable to 0 and maxSurge to 0 to avoid extra pods.

Wrong approach:

spec:
strategy:
rollingUpdate:
maxUnavailable: 0
maxSurge: 0

Correct approach:

spec:
strategy:
rollingUpdate:
maxUnavailable: 1
maxSurge: 1

Root cause:

Not allowing any pod replacement concurrency causes the update to stall or fail.

#3 Ignoring readiness probes leading to premature pod deletion.

Wrong approach:

No readiness probe defined in pod spec.

Correct approach:

Add readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 10

Root cause:

Without readiness probes, Kubernetes may delete old pods before new pods are ready, causing downtime.

Key Takeaways

Rolling update strategy updates app pods gradually to keep the service available during changes.

It relies on Kubernetes deployments managing pod replacement with controls like max surge and max unavailable.

Only changes to the pod template trigger rolling updates, not all deployment changes.

Readiness probes and Pod Disruption Budgets are critical to smooth and safe rolling updates.

Understanding rollback and failure handling ensures stability and quick recovery in production.