Enrich processor in Elasticsearch - Step-by-Step Execution

Concept Flow - Enrich processor

Start ingest pipeline

↓

Receive document

↓

Extract key field

↓

Lookup enrich index with key

↓

Match found?

No→Skip enrich

Yes↓

Merge enrich data into document

↓

Pass enriched document to next processor

↓

End ingest pipeline

The enrich processor takes a field from the incoming document, looks it up in an enrich index, and merges matching data into the document before continuing.

Execution Sample

Elasticsearch

PUT _ingest/pipeline/enrich-pipeline
{
  "processors": [
    {
      "enrich": {
        "policy_name": "user-policy",
        "field": "user_id",
        "target_field": "user_info"
      }
    }
  ]
}

Defines an ingest pipeline that enriches documents by looking up user info based on user_id and adds it under user_info field.

Execution Table

Step	Action	Input Document	Lookup Key	Enrich Index Result	Output Document
1	Receive document	{"user_id": "123", "event": "login"}	123	-	{"user_id": "123", "event": "login"}
2	Lookup enrich index	-	123	{"name": "Alice", "role": "admin"}	-
3	Merge enrich data	-	-	-	{"user_id": "123", "event": "login", "user_info": {"name": "Alice", "role": "admin"}}
4	Pass to next processor	-	-	-	{"user_id": "123", "event": "login", "user_info": {"name": "Alice", "role": "admin"}}
5	End pipeline	-	-	-	Pipeline finished with enriched document

💡 Pipeline ends after enriching document or skipping if no match found.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
document	{"user_id": "123", "event": "login"}	{"user_id": "123", "event": "login"}	{"user_id": "123", "event": "login"}	{"user_id": "123", "event": "login", "user_info": {"name": "Alice", "role": "admin"}}	{"user_id": "123", "event": "login", "user_info": {"name": "Alice", "role": "admin"}}
lookup_key	-	123	123	123	123
enrich_result	-	-	{"name": "Alice", "role": "admin"}	{"name": "Alice", "role": "admin"}	{"name": "Alice", "role": "admin"}

Key Moments - 3 Insights

What happens if the enrich index has no matching entry for the lookup key?

If no match is found (see execution_table step 2), the enrich processor skips merging and passes the original document unchanged to the next processor.

How does the enrich processor know which field to use for lookup?

The processor uses the 'field' parameter from the pipeline definition (shown in execution_sample) to extract the lookup key from the incoming document.

Where does the enriched data get added in the document?

The enriched data is merged under the 'target_field' specified in the pipeline (here 'user_info'), as shown in execution_table step 3.

Visual Quiz - 3 Questions

Look at the execution_table, what is the value of 'lookup_key' after Step 1?

A 123

B -

C Alice

D {"name": "Alice", "role": "admin"}

💡 Hint

Check variable_tracker column 'After Step 1' for 'lookup_key'

At which step does the document get enriched with user info?

A Step 1

B Step 3

C Step 2

D Step 4

💡 Hint

Look at execution_table 'Action' and 'Output Document' columns

If the enrich index had no matching entry, what would happen to the output document?

A The pipeline would fail

B It would be enriched with empty data

C It would remain unchanged

D The document would be deleted

💡 Hint

Refer to key_moments explanation about no match scenario

Concept Snapshot

Enrich processor syntax:
{
  "enrich": {
    "policy_name": "policy",
    "field": "lookup_field",
    "target_field": "field_to_add"
  }
}

Behavior: Looks up a value from an enrich index using 'field' from the document, merges found data under 'target_field'.
If no match, document passes unchanged.

Full Transcript

The enrich processor in Elasticsearch ingest pipelines takes a field from the incoming document, uses it as a key to look up data in an enrich index, and merges the found data into the document under a target field. The process starts by receiving the document, extracting the lookup key, querying the enrich index, and if a match is found, merging the data. If no match is found, the document continues unchanged. This allows adding extra information to documents during ingestion based on external data.