Index lifecycle management in Elasticsearch - Step-by-Step Execution

Concept Flow - Index lifecycle management

Create Index

↓

Attach Lifecycle Policy

↓

Index Enters Hot Phase

↓

Index Data Ingested

↓

Transition to Warm Phase

↓

Optimize and Shrink Index

↓

Transition to Cold Phase

↓

Read-Only, Less Frequent Access

↓

Transition to Delete Phase

↓

Delete Index

↓

End of Lifecycle

The index lifecycle starts with creation, then moves through phases (hot, warm, cold) with actions, ending in deletion.

Execution Sample

Elasticsearch

PUT /my-index-000001
{
  "settings": {
    "index.lifecycle.name": "my_policy"
  }
}

This creates an index and attaches a lifecycle policy named 'my_policy' to manage its phases.

Execution Table

Step	Action	Phase	Index State	Result
1	Create index with policy	N/A	Index created	Index 'my-index-000001' created with lifecycle policy 'my_policy'
2	Index enters hot phase	Hot	Writable, accepting data	Index is actively receiving data
3	Data ingestion	Hot	Growing index size	Documents are indexed
4	Transition to warm phase	Warm	Read-only, optimized	Index is shrunk and optimized
5	Transition to cold phase	Cold	Read-only, less frequent access	Index stored on cheaper hardware
6	Transition to delete phase	Delete	Marked for deletion	Index scheduled for deletion
7	Delete index	Delete	Deleted	Index 'my-index-000001' removed
8	End	N/A	No index	Lifecycle complete, index no longer exists

💡 Index deleted, lifecycle management ends

Variable Tracker

Variable	Start	After Step 2	After Step 4	After Step 6	Final
Index Phase	N/A	Hot	Warm	Delete	Deleted
Index State	Created	Writable	Read-only Optimized	Marked for Deletion	Deleted
Index Size	0	Growing	Stable	Stable	N/A

Key Moments - 3 Insights

Why does the index become read-only in the warm phase?

Because in the warm phase the index is optimized and shrunk to save resources, so it is set to read-only to prevent new data ingestion, as shown in execution_table row 4.

What triggers the transition from hot to warm phase?

The lifecycle policy defines conditions like index age or size; when met, the index transitions automatically, as seen between steps 3 and 4 in the execution_table.

Why is the index deleted at the end of lifecycle?

To free up storage and resources, the lifecycle policy deletes the index after it is no longer needed, as shown in execution_table steps 6 and 7.

Visual Quiz - 3 Questions

Look at the execution_table, what is the index state after step 4?

A Marked for deletion

B Writable and accepting data

C Read-only and optimized

D Deleted

💡 Hint

Check the 'Index State' column at step 4 in the execution_table

At which step does the index stop accepting new data?

A Step 4

B Step 2

C Step 3

D Step 6

💡 Hint

Look for when the index becomes read-only in the execution_table

If the lifecycle policy is removed, what would happen to the index lifecycle?

A Index would still transition phases automatically

B Index would remain in hot phase indefinitely

C Index would be deleted immediately

D Index would become read-only immediately

💡 Hint

Refer to the concept_flow where lifecycle policy controls phase transitions

Concept Snapshot

Index lifecycle management controls index phases: hot (write), warm (read-only, optimize), cold (less access), delete (remove).
Attach a policy to automate transitions based on age or size.
Phases save resources and manage data efficiently.
Deletion frees storage when data is no longer needed.

Full Transcript

Index lifecycle management in Elasticsearch automates how an index moves through different phases: hot, warm, cold, and delete. It starts by creating an index and attaching a lifecycle policy. The index begins in the hot phase where it accepts writes and data ingestion happens. When conditions like age or size are met, it transitions to the warm phase, becoming read-only and optimized to save resources. Later, it moves to the cold phase for less frequent access, stored on cheaper hardware. Finally, the index enters the delete phase where it is removed to free storage. This process helps manage data efficiently without manual intervention.