Hot-warm-cold architecture in Elasticsearch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this hot-warm-cold node allocation query?

Given an Elasticsearch cluster with hot, warm, and cold nodes, what will this query return?

Elasticsearch

GET /_cat/allocation?v

# Assume the cluster has 3 hot nodes, 2 warm nodes, and 1 cold node.
# The query shows disk usage per node.

A Returns an error because _cat/allocation does not support node tiers.

B Lists all nodes with their disk usage, showing hot nodes with highest disk usage.

C Returns only cold nodes with zero disk usage.

D Returns only hot nodes with their disk usage.

💡 Hint

The _cat/allocation API shows allocation info for all nodes by default.

The _cat/allocation API lists all nodes in the cluster with their disk usage and shard allocation. It does not filter by node tier unless specified.

🧠 Conceptual

intermediate

Which node tier is best suited for storing frequently updated data?

In a hot-warm-cold Elasticsearch architecture, which node tier should store data that is updated frequently and requires fast access?

A Hot nodes, because they provide the fastest access and indexing.

B Warm nodes, because they balance storage and performance.

C Cold nodes, because they have the most storage capacity.

D Any node tier, because Elasticsearch automatically balances updates.

💡 Hint

Think about which nodes handle indexing and fast queries.

Hot nodes are designed for fast indexing and search of recent data that changes frequently.

🔧 Debug

advanced

Why does this index not move to the warm tier as expected?

An index lifecycle policy is set to move indices from hot to warm after 7 days, but indices remain on hot nodes. What is the likely cause?

Elasticsearch

PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "7d"
          }
        }
      },
      "warm": {
        "actions": {
          "allocate": {
            "require": {
              "data": "warm"
            }
          }
        }
      }
    }
  }
}

A The warm nodes are not tagged with the attribute 'data=warm'.

B The rollover max_age is too short, causing indices to stay hot.

C The allocate action is missing the 'include' key.

D The policy is missing a cold phase, so warm phase is ignored.

💡 Hint

Check node attributes and allocation filtering.

If warm nodes lack the required attribute 'data=warm', allocation cannot move shards there.

📝 Syntax

advanced

Identify the syntax error in this ILM policy snippet for cold phase allocation

Which option contains the correct syntax to allocate indices to cold nodes in the cold phase?

Elasticsearch

PUT _ilm/policy/cold_policy
{
  "policy": {
    "phases": {
      "cold": {
        "min_age": "30d",
        "actions": {
          "allocate": {
            "require": {
              "data": "cold"
            }
          }
        }
      }
    }
  }
}

A { "allocate": { "require": [ "data": "cold" ] } }

B { "allocate": { "include": { "data": "cold" } } }

C { "allocate": { "require": { "data": "cold" } } }

D { "allocate": { "require": { data: "cold" } } }

💡 Hint

JSON keys must be strings and use curly braces for objects.

Option C uses correct JSON object syntax with string keys and nested objects.

🚀 Application

expert

How many shards will be on cold nodes after 60 days with this ILM policy?

An index has 10 primary shards and 1 replica. The ILM policy moves data from hot to warm at 7 days, then to cold at 30 days. After 60 days, how many shards (primary + replica) are allocated on cold nodes?

Elasticsearch

ILM policy phases:
- hot: 0-7 days
- warm: 7-30 days
- cold: 30+ days

Index settings:
- number_of_shards: 10
- number_of_replicas: 1

A 30 shards (10 primary + 10 replica + 10 from warm nodes) on cold nodes

B 10 shards (only primary) on cold nodes

C 0 shards on cold nodes because replicas stay on warm nodes

D 20 shards (10 primary + 10 replica) on cold nodes

💡 Hint

Replicas move with primaries during ILM phase transitions.

Both primary and replica shards move to cold nodes after 30 days, so total shards are 10 primaries + 10 replicas = 20.

Practice

1. What is the main purpose of the hot-warm-cold architecture in Elasticsearch?

easy

A. To encrypt data at rest and in transit

B. To store recent data on fast nodes and older data on slower, cheaper nodes

C. To backup data to external storage automatically

D. To replicate data across multiple clusters for high availability

Solution

1. Step 1: Understand the architecture purpose

   The hot-warm-cold architecture is designed to optimize storage costs and performance by placing recent data on fast nodes and older data on slower, cheaper nodes.

2. Step 2: Match the purpose to options

   To store recent data on fast nodes and older data on slower, cheaper nodes correctly describes this purpose, while other options describe different Elasticsearch features.

3. Final Answer:

   To store recent data on fast nodes and older data on slower, cheaper nodes -> Option B

4. Quick Check:

   Hot-warm-cold architecture = store data by age and speed [OK]

Hint: Remember: hot = fast recent, cold = slow old data [OK]

Common Mistakes:

• Confusing hot-warm-cold with backup or replication
• Thinking it encrypts data automatically
• Assuming it manages cluster replication

2. Which Elasticsearch feature is used to automate moving data between hot, warm, and cold phases?

easy

A. Snapshot and Restore

B. Document Level Security

C. Index Lifecycle Management (ILM)

D. Cross-cluster Search

Solution

1. Step 1: Identify automation for data phase movement

   Index Lifecycle Management (ILM) automates moving indices through hot, warm, and cold phases based on policies.

2. Step 2: Compare other features

   Snapshot and Restore handles backups, Cross-cluster Search queries multiple clusters, and Document Level Security controls access, so they don't automate data movement.

3. Final Answer:

   Index Lifecycle Management (ILM) -> Option C

4. Quick Check:

   ILM automates data phase transitions [OK]

Hint: ILM = automates index phase changes [OK]

Common Mistakes:

• Choosing Snapshot instead of ILM
• Confusing security features with lifecycle management
• Thinking cross-cluster search manages data phases

3. Given this ILM policy snippet, what phase will the index move to after 30 days?

{
  "phases": {
    "hot": {"min_age": "0d"},
    "warm": {"min_age": "7d"},
    "cold": {"min_age": "30d"}
  }
}

medium

A. Cold phase

B. Warm phase

C. Hot phase

D. Delete phase

Solution

1. Step 1: Analyze min_age values for phases

   The policy defines hot from 0 days, warm from 7 days, and cold from 30 days.

2. Step 2: Determine phase after 30 days

   After 30 days, the index reaches the cold phase because its min_age is 30 days, which is the threshold for cold.

3. Final Answer:

   Cold phase -> Option A

4. Quick Check:

   30 days = cold phase start [OK]

Hint: Check min_age values to find current phase [OK]

Common Mistakes:

• Choosing warm phase after 30 days
• Confusing delete phase with cold phase
• Ignoring min_age thresholds

4. You wrote this ILM policy but your index never moves to the warm phase:

{
  "phases": {
    "hot": {"min_age": "0d"},
    "warm": {"min_age": "10d"}
  }
}

What is the likely problem?

medium

A. The index size is too small to trigger rollover

B. The warm phase min_age is too low

C. The warm phase is missing an allocation action

D. The policy lacks a cold phase

Solution

1. Step 1: Understand ILM phase transition requirements

   For an index to move from hot to warm, rollover conditions like size or age must be met.

2. Step 2: Identify missing trigger

   If the index size is too small, rollover won't happen, so the index stays in hot phase and never moves to warm.

3. Final Answer:

   The index size is too small to trigger rollover -> Option A

4. Quick Check:

   Small index size blocks rollover and phase move [OK]

Hint: Check rollover conditions to enable phase change [OK]

Common Mistakes:

• Assuming missing allocation causes no move
• Thinking warm phase min_age is too low
• Believing cold phase is required to move to warm

5. You want to optimize storage costs by moving indices older than 60 days to cold nodes and delete indices older than 90 days. Which ILM policy snippet correctly implements this?

hard

A. { "phases": { "hot": {"min_age": "0d"}, "warm": {"min_age": "30d"}, "cold": {"min_age": "90d"}, "delete": {"min_age": "90d"} } }

B. { "phases": { "hot": {"min_age": "0d"}, "warm": {"min_age": "30d"}, "delete": {"min_age": "60d"} } }

C. { "phases": { "hot": {"min_age": "0d"}, "warm": {"min_age": "60d"}, "cold": {"min_age": "90d"}, "delete": {"min_age": "120d"} } }

D. { "phases": { "hot": {"min_age": "0d"}, "cold": {"min_age": "60d"}, "delete": {"min_age": "90d"} } }

Solution

1. Step 1: Identify required phase ages

   Indices older than 60 days should move to cold, and older than 90 days should be deleted.

2. Step 2: Match policy phases to requirements

   { "phases": { "hot": {"min_age": "0d"}, "cold": {"min_age": "60d"}, "delete": {"min_age": "90d"} } } has hot at 0d, cold at 60d, and delete at 90d, matching the requirements exactly.

3. Final Answer:

   { "phases": { "hot": {"min_age": "0d"}, "cold": {"min_age": "60d"}, "delete": {"min_age": "90d"} } } -> Option D

4. Quick Check:

   60d cold and 90d delete phases match [OK]

Hint: Match min_age exactly to your data lifecycle needs [OK]

Common Mistakes:

• Adding unnecessary warm phase with wrong min_age
• Setting delete phase too early
• Skipping cold phase before delete