Index lifecycle management in Elasticsearch

Introduction

Index lifecycle management helps you automatically manage your data as it grows old or becomes less important. It saves space and keeps your search fast.

You want to delete old logs after a certain time to save storage.

You want to move data to cheaper storage when it is not used often.

You want to keep recent data on fast disks for quick searching.

You want to automatically create new indexes for new data.

You want to reduce manual work managing many indexes.

Syntax

Elasticsearch

PUT _ilm/policy/<policy_name>
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "50gb",
            "max_age": "30d"
          }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

The policy defines phases like hot, warm, cold, and delete.

Each phase has actions like rollover, shrink, or delete to manage indexes automatically.

Examples

This policy rolls over the index after 7 days and deletes it after 30 days.

Elasticsearch

PUT _ilm/policy/log_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "7d"
          }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

This policy moves data to warm nodes after 30 days and freezes it after 60 days to save resources.

Elasticsearch

PUT _ilm/policy/archive_policy
{
  "policy": {
    "phases": {
      "warm": {
        "min_age": "30d",
        "actions": {
          "allocate": {
            "require": {
              "box_type": "warm"
            }
          }
        }
      },
      "cold": {
        "min_age": "60d",
        "actions": {
          "freeze": {}
        }
      }
    }
  }
}

Sample Program

This example creates a policy that rolls over indexes daily or after 10GB and deletes them after 7 days. Then it retrieves the policy to confirm.

Elasticsearch

PUT _ilm/policy/sample_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_size": "10gb",
            "max_age": "1d"
          }
        }
      },
      "delete": {
        "min_age": "7d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

GET _ilm/policy/sample_policy

Important Notes

Make sure your index template uses the lifecycle policy by setting index.lifecycle.name.

Rollover requires a write alias to work properly.

Phases run automatically based on index age and size.

Summary

Index lifecycle management automates index handling to save space and improve performance.

Policies define phases with actions like rollover and delete.

Use ILM to reduce manual work and keep your Elasticsearch cluster healthy.