How to Use Date Processor in Elasticsearch for Date Parsing

Use the date processor in an Elasticsearch ingest pipeline to parse date strings into date objects. Define the field containing the date string and specify the formats to match the input date pattern.

Syntax

The date processor syntax includes specifying the field to parse, the target_field where the parsed date will be stored (optional), and the formats array to define acceptable date patterns.

• field: The source field with the date string.
• target_field: The field to store the parsed date (defaults to field if omitted).
• formats: An array of date format patterns to parse the input.

json

{
  "date": {
    "field": "source_date",
    "target_field": "parsed_date",
    "formats": ["yyyy-MM-dd'T'HH:mm:ssZ", "yyyy-MM-dd"]
  }
}

Example

This example shows how to create an ingest pipeline with a date processor that parses a date string from the timestamp field and stores it back in the same field as a date object.

json

PUT _ingest/pipeline/date_pipeline
{
  "description": "Parse date string to date object",
  "processors": [
    {
      "date": {
        "field": "timestamp",
        "formats": ["yyyy-MM-dd HH:mm:ss", "ISO8601"]
      }
    }
  ]
}

POST _ingest/pipeline/date_pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "timestamp": "2024-06-15 14:30:00"
      }
    }
  ]
}

Output

{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_id": "_id",
        "_source": {
          "timestamp": "2024-06-15T14:30:00.000Z"
        },
        "_ingest": {
          "timestamp": "2024-06-15T14:30:00.000Z"
        }
      }
    }
  ]
}

Common Pitfalls

Common mistakes when using the date processor include:

• Not matching the input date string format exactly with the formats array, causing parsing failures.
• Forgetting to specify the correct field name where the date string exists.
• Assuming the processor changes the original field without setting target_field properly.
• Not handling time zones correctly in the date format.

json

/* Wrong: format does not match input date string */
{
  "date": {
    "field": "timestamp",
    "formats": ["yyyy/MM/dd"]
  }
}

/* Right: format matches input date string */
{
  "date": {
    "field": "timestamp",
    "formats": ["yyyy-MM-dd HH:mm:ss"]
  }
}

Quick Reference

Property	Description
field	The source field containing the date string to parse.
target_field	Optional field to store the parsed date; defaults to the source field.
formats	Array of date format patterns to parse the input string.
timezone	Optional timezone to apply if the input date string lacks one.
ignore_failure	Optional flag to skip processor on failure without error.

Key Takeaways

Use the date processor in an ingest pipeline to convert date strings to date objects.

Always match the input date format exactly in the formats array to avoid parsing errors.

Specify the correct source field and optionally a target field for the parsed date.

Use the simulate API to test your pipeline before applying it to real data.

Handle time zones explicitly if your date strings do not include them.