Why Rolling upgrades in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if you could upgrade your system without anyone noticing a thing?

The Scenario

Imagine you have a big library where people come to read books all day. Now, you want to replace some old shelves with new ones without closing the library. If you stop everything to replace shelves, visitors get frustrated and leave.

The Problem

Manually upgrading all parts at once means shutting down the whole system. This causes downtime, unhappy users, and lost data. It's like closing the library completely, making everyone wait until you finish.

The Solution

Rolling upgrades let you update parts one by one while the rest keep working. It's like replacing shelves section by section without closing the library, so visitors can still read books without interruption.

Before vs After

✗ Before

Stop all nodes
Upgrade software
Restart all nodes

✓ After

Upgrade node 1
Wait for it to join
Upgrade node 2
Repeat until done

What It Enables

Rolling upgrades enable continuous service with zero downtime during updates, keeping users happy and data safe.

Real Life Example

A popular website updates its search engine software without taking the site offline, so users can keep searching without noticing any interruption.

Key Takeaways

Manual upgrades cause full downtime and disrupt users.

Rolling upgrades update nodes one at a time, keeping service live.

This approach ensures smooth, safe, and continuous operation during upgrades.

Practice

1. What is the main purpose of performing a rolling upgrade in Elasticsearch?

easy

A. To disable the cluster permanently during upgrade

B. To upgrade all nodes simultaneously for faster updates

C. To upgrade nodes one by one without stopping the entire cluster

D. To delete old data before upgrading

Solution

1. Step 1: Understand rolling upgrade concept

   A rolling upgrade updates nodes one at a time to keep the cluster running.

2. Step 2: Compare options

   Only To upgrade nodes one by one without stopping the entire cluster describes upgrading nodes one by one without stopping the cluster.

3. Final Answer:

   To upgrade nodes one by one without stopping the entire cluster -> Option C

4. Quick Check:

   Rolling upgrade = upgrade nodes individually [OK]

Hint: Rolling upgrade means upgrading nodes one by one [OK]

Common Mistakes:

• Thinking all nodes upgrade at once
• Confusing rolling upgrade with cluster shutdown
• Assuming data is deleted during upgrade

2. Which command is recommended to disable shard allocation before starting a rolling upgrade?

easy

A. PUT /_cluster/settings {"persistent": {"cluster.routing.allocation.enable": "none"}}

B. POST /_cluster/disable_shards

C. GET /_cluster/settings {"allocation": "disable"}

D. DELETE /_cluster/shards

Solution

1. Step 1: Identify correct syntax to disable shard allocation

   The correct way is to update cluster settings with PUT and set allocation to "none".

2. Step 2: Check options

   Only PUT /_cluster/settings {"persistent": {"cluster.routing.allocation.enable": "none"}} uses the correct HTTP method, endpoint, and JSON body.

3. Final Answer:

   PUT /_cluster/settings {"persistent": {"cluster.routing.allocation.enable": "none"}} -> Option A

4. Quick Check:

   Disable shard allocation = PUT cluster settings with allocation none [OK]

Hint: Use PUT with cluster settings and allocation none to disable shards [OK]

Common Mistakes:

• Using wrong HTTP method like POST or GET
• Wrong endpoint or missing persistent key
• Trying to delete shards instead of disabling allocation

3. Given the following sequence during a rolling upgrade:

1. Disable shard allocation
2. Upgrade node 1
3. Upgrade node 2
4. Enable shard allocation

What is the expected cluster behavior after step 4?

medium

A. The cluster will stop accepting new data

B. The cluster will rebalance shards across all nodes

C. The cluster will delete old shards permanently

D. The cluster will remain unbalanced with shards stuck

Solution

1. Step 1: Understand shard allocation states

   Disabling shard allocation prevents shard movement during upgrade; enabling it allows rebalancing.

2. Step 2: Analyze cluster behavior after enabling allocation

   After enabling, the cluster redistributes shards to balance load.

3. Final Answer:

   The cluster will rebalance shards across all nodes -> Option B

4. Quick Check:

   Enable allocation = rebalance shards [OK]

Hint: Enabling shard allocation triggers shard rebalancing [OK]

Common Mistakes:

• Thinking cluster stops accepting data
• Assuming shards get deleted
• Believing shards remain stuck after enabling allocation

4. You ran a rolling upgrade but forgot to disable shard allocation first. What problem might occur?

medium

A. Shards may move during upgrade causing data loss or instability

B. The upgrade will fail immediately with syntax error

C. The cluster will automatically disable allocation for you

D. Nothing happens; upgrade proceeds safely

Solution

1. Step 1: Understand shard allocation role during upgrade

   Disabling allocation prevents shards from moving and keeps data safe during node restarts.

2. Step 2: Consequence of not disabling allocation

   If not disabled, shards may relocate while nodes restart, risking data loss or cluster instability.

3. Final Answer:

   Shards may move during upgrade causing data loss or instability -> Option A

4. Quick Check:

   Not disabling allocation risks shard movement and instability [OK]

Hint: Always disable shard allocation before upgrade to avoid shard moves [OK]

Common Mistakes:

• Expecting upgrade to fail with syntax error
• Assuming cluster disables allocation automatically
• Thinking upgrade is safe without disabling allocation

5. During a rolling upgrade, you want to ensure minimal downtime and data safety. Which sequence of actions is best practice?

hard

A. Upgrade all nodes simultaneously -> Disable shard allocation -> Enable shard allocation -> Restart cluster

B. Restart cluster -> Disable shard allocation -> Upgrade nodes one by one -> Enable shard allocation

C. Enable shard allocation -> Upgrade nodes one by one -> Disable shard allocation -> Verify cluster health

D. Disable shard allocation -> Upgrade nodes one by one -> Enable shard allocation -> Verify cluster health

Solution

1. Step 1: Identify correct upgrade steps

   Best practice is to disable shard allocation first to prevent shard moves, then upgrade nodes one by one.

2. Step 2: Finalize upgrade process

   After upgrading all nodes, enable shard allocation to rebalance shards and verify cluster health to confirm stability.

3. Final Answer:

   Disable shard allocation -> Upgrade nodes one by one -> Enable shard allocation -> Verify cluster health -> Option D

4. Quick Check:

   Disable allocation, upgrade nodes, enable allocation, check health [OK]

Hint: Disable allocation first, upgrade nodes, then enable allocation [OK]

Common Mistakes:

• Upgrading all nodes at once causing downtime
• Enabling allocation before upgrade completion
• Restarting cluster unnecessarily