Enrich processor in Elasticsearch - Mini Project: Build & Apply

Enrich Processor in Elasticsearch

📖 Scenario: You work in a company that stores customer orders in Elasticsearch. You want to add extra customer details to each order automatically using Elasticsearch's enrich processor.

🎯 Goal: Build an Elasticsearch ingest pipeline that enriches order documents with customer information from a separate customer index.

📋 What You'll Learn

Create an enrich policy named customer-policy that matches on customer_id from the customers index.

Create an ingest pipeline named order-enrich-pipeline that uses the enrich processor with the customer-policy.

Test the pipeline by simulating an ingest of an order document with a customer_id.

Print the enriched order document showing added customer details.

💡 Why This Matters

🌍 Real World

Enrich processor helps automatically add related data to documents during ingest, saving time and avoiding manual joins.

💼 Career

Understanding enrich processors is useful for Elasticsearch engineers and developers working on data pipelines and search applications.

1

Create an enrich policy

Write a JSON request to create an enrich policy called customer-policy that uses the match type on the customer_id field from the customers index.

Elasticsearch

{
  "policy": {
    "match": {
      "indices": ["customers"],
      "match_field": "customer_id",
      "enrich_fields": ["name", "email"]
    }
  }
}

# Your code here

Need a hint?

Use the PUT _enrich/policy/customer-policy API with a JSON body defining the match type, indices, match_field, and enrich_fields.

2

Create an ingest pipeline with enrich processor

Write a JSON request to create an ingest pipeline named order-enrich-pipeline that uses the enrich processor with the policy customer-policy and targets the field customer_id. The enriched data should be added under the field customer_info.

Elasticsearch

{
  "policy": {
    "match": {
      "indices": ["customers"],
      "match_field": "customer_id",
      "enrich_fields": ["name", "email"]
    }
  }
}

# Your code here to create the ingest pipeline

Need a hint?

Use the PUT _ingest/pipeline/order-enrich-pipeline API with a processors array containing an enrich processor configured with the policy name, field, and target_field.

3

Simulate ingesting an order document

Write a JSON request to simulate ingesting an order document with customer_id set to 123 through the order-enrich-pipeline. Include an order_id field with value abc-001.

Elasticsearch

{
  "policy": {
    "match": {
      "indices": ["customers"],
      "match_field": "customer_id",
      "enrich_fields": ["name", "email"]
    }
  }
}

PUT _ingest/pipeline/order-enrich-pipeline
{
  "processors": [
    {
      "enrich": {
        "policy_name": "customer-policy",
        "field": "customer_id",
        "target_field": "customer_info",
        "ignore_missing": true
      }
    }
  ]
}

POST _ingest/pipeline/order-enrich-pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "order_id": "abc-001",
        "customer_id": 123
      }
    }
  ]
}

# Your code here

Need a hint?

Use the POST _ingest/pipeline/order-enrich-pipeline/_simulate API with a docs array containing the order document.

4

Print the enriched order document

Write a command to print the enriched order document returned by the simulate API, showing the order_id, customer_id, and the added customer_info fields.

Elasticsearch

{
  "policy": {
    "match": {
      "indices": ["customers"],
      "match_field": "customer_id",
      "enrich_fields": ["name", "email"]
    }
  }
}

PUT _ingest/pipeline/order-enrich-pipeline
{
  "processors": [
    {
      "enrich": {
        "policy_name": "customer-policy",
        "field": "customer_id",
        "target_field": "customer_info",
        "ignore_missing": true
      }
    }
  ]
}

POST _ingest/pipeline/order-enrich-pipeline/_simulate
{
  "docs": [
    {
      "_source": {
        "order_id": "abc-001",
        "customer_id": 123
      }
    }
  ]
}

# Your code here to print the enriched document

Need a hint?

Look at the response from the simulate API. It contains the enriched document under docs[0]._source. Print or display this JSON to see the added customer_info.