Index lifecycle management in Elasticsearch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Index Lifecycle Management (ILM) in Elasticsearch?

ILM is a feature in Elasticsearch that automates the management of index lifecycles, such as creation, rollover, and deletion, based on defined policies to optimize storage and performance.

beginner

Name the typical phases in an Elasticsearch ILM policy.

The typical ILM phases are: hot (indexing and searching), warm (less frequent access, optimized for cost), cold (rarely accessed, low cost), and delete (removes old data).

intermediate

What is the purpose of the 'rollover' action in ILM?

The rollover action creates a new index when the current index reaches a size, age, or document count threshold, helping to keep indexes manageable and performant.

intermediate

How does ILM help with storage cost optimization?

ILM moves older indexes to less expensive hardware or storage tiers during the warm and cold phases, reducing storage costs while keeping data accessible if needed.

beginner

What happens during the 'delete' phase in an ILM policy?

During the delete phase, Elasticsearch automatically deletes indexes that are no longer needed, freeing up storage space and keeping the cluster clean.

Which ILM phase is primarily used for active indexing and searching?

A delete

B hot

C cold

D warm

The hot phase is where active indexing and searching happen.

What does the rollover action in ILM do?

A Deletes old indexes

B Moves indexes to cold storage

C Creates a new index when thresholds are met

D Optimizes index shards

Rollover creates a new index when size, age, or document count limits are reached.

In which ILM phase are indexes moved to less expensive storage but remain searchable?

A warm

B delete

C hot

D indexing

The warm phase moves indexes to cheaper storage while keeping them searchable.

What is the main goal of the delete phase in ILM?

A Free up storage by removing old indexes

B Improve search speed

C Create new indexes

D Move data to hot storage

The delete phase removes old indexes to free storage space.

Which of the following is NOT a typical ILM phase?

A cold

B warm

C hot

D freeze

Freeze is not a standard ILM phase; the main phases are hot, warm, cold, and delete.

Explain the purpose and benefits of using Index Lifecycle Management in Elasticsearch.

Think about how ILM helps manage data as it ages.

Describe the rollover action and when it is triggered in an ILM policy.

Consider what happens when an index gets too big or old.

Practice

1. What is the main purpose of Index Lifecycle Management (ILM) in Elasticsearch?

easy

A. To automate index handling and improve cluster performance

B. To manually create indexes for each data type

C. To backup data to external storage

D. To encrypt index data for security

Solution

1. Step 1: Understand ILM's role

   ILM automates managing indexes through their lifecycle phases.

2. Step 2: Identify the benefit

   Automation helps save space and improves cluster health and performance.

3. Final Answer:

   To automate index handling and improve cluster performance -> Option A

4. Quick Check:

   ILM automates index management = A [OK]

Hint: ILM automates index tasks to keep Elasticsearch healthy [OK]

Common Mistakes:

• Thinking ILM is for manual index creation
• Confusing ILM with backup or encryption
• Assuming ILM only deletes indexes

2. Which of the following is the correct way to define a rollover action in an ILM policy JSON?

easy

A. "rollover": { "max_size": "50gb", "max_age": "30d" }

B. "rollover": "true"

C. "rollover": { "size": "50gb" }

D. "rollover": { "max_documents": 1000 }

Solution

1. Step 1: Check rollover action syntax

   Rollover requires conditions like max_size and max_age in a JSON object.

2. Step 2: Validate options

   "rollover": { "max_size": "50gb", "max_age": "30d" } correctly uses max_size and max_age keys with proper values.

3. Final Answer:

   "rollover": { "max_size": "50gb", "max_age": "30d" } -> Option A

4. Quick Check:

   Rollover needs max_size and max_age keys = A [OK]

Hint: Rollover needs max_size and max_age keys in JSON [OK]

Common Mistakes:

• Using string 'true' instead of object for rollover
• Missing max_age or max_size keys
• Using incorrect key names like 'size'

3. Given this ILM policy snippet:

{"phases": {"hot": {"actions": {"rollover": {"max_size": "10gb"}}}, "delete": {"min_age": "90d", "actions": {"delete": {}}}}}

What happens to an index after it reaches 10GB?

medium

A. It archives data to external storage

B. It rolls over to a new index and continues indexing

C. It stops accepting new data but stays

D. It is immediately deleted

Solution

1. Step 1: Analyze rollover action in hot phase

   The policy rolls over the index when it reaches 10GB.

2. Step 2: Understand delete phase timing

   Deletion happens only after 90 days, so not immediate.

3. Final Answer:

   It rolls over to a new index and continues indexing -> Option B

4. Quick Check:

   Rollover at 10GB, delete after 90d = D [OK]

Hint: Rollover triggers at size, delete waits min_age [OK]

Common Mistakes:

• Confusing rollover with immediate deletion
• Ignoring min_age delay for delete phase
• Assuming data stops indexing after rollover

4. You have this ILM policy JSON:

{"phases": {"hot": {"actions": {"rollover": {"max_size": "20gb"}}}, "warm": {"min_age": "10d", "actions": {"shrink": {"number_of_shards": 1}}}}}

After applying, the shrink action never runs. What is the likely cause?

medium

A. Rollover max_size is too small

B. Shrink action requires delete phase

C. Shrink action must be in hot phase

D. The index has not reached 10 days old yet

Solution

1. Step 1: Check warm phase min_age

   Shrink runs only after index is at least 10 days old.

2. Step 2: Confirm index age

   If index age is less than 10 days, shrink won't trigger yet.

3. Final Answer:

   The index has not reached 10 days old yet -> Option D

4. Quick Check:

   min_age controls action timing = C [OK]

Hint: Check min_age before expecting warm phase actions [OK]

Common Mistakes:

• Assuming shrink needs delete phase
• Changing rollover size unrelated to shrink
• Placing shrink in wrong phase

5. You want to create an ILM policy that:
- Rolls over an index at 30GB or 15 days
- Moves the index to warm phase after rollover
- Deletes the index 60 days after rollover

Which policy configuration correctly implements this?

hard

A. {"phases": {"hot": {"actions": {"rollover": {"max_size": "30gb"}}}, "warm": {"min_age": "30d", "actions": {}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}}

B. {"phases": {"hot": {"actions": {"rollover": {"max_age": "15d"}}}, "warm": {"min_age": "15d", "actions": {"shrink": {}}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}}

C. {"phases": {"hot": {"actions": {"rollover": {"max_size": "30gb", "max_age": "15d"}}}, "warm": {"min_age": "15d", "actions": {}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}}

D. {"phases": {"hot": {"actions": {"rollover": {"max_size": "15gb", "max_age": "30d"}}}, "warm": {"min_age": "15d", "actions": {}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}}

Solution

1. Step 1: Verify rollover conditions

   {"phases": {"hot": {"actions": {"rollover": {"max_size": "30gb", "max_age": "15d"}}}, "warm": {"min_age": "15d", "actions": {}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}} uses max_size 30gb and max_age 15d, matching requirements.

2. Step 2: Check warm and delete phases

   Warm phase starts at 15d (after rollover), delete at 60d, as required.

3. Step 3: Confirm other options

   Other options have wrong sizes, ages, or missing actions.

4. Final Answer:

   {"phases": {"hot": {"actions": {"rollover": {"max_size": "30gb", "max_age": "15d"}}}, "warm": {"min_age": "15d", "actions": {}}, "delete": {"min_age": "60d", "actions": {"delete": {}}}}} -> Option C

5. Quick Check:

   Rollover at 30gb/15d, warm at 15d, delete at 60d = B [OK]

Hint: Match rollover size and age, then warm and delete min_age [OK]

Common Mistakes:

• Swapping max_size and max_age values
• Setting warm min_age too late
• Missing delete action or wrong min_age