Percolate queries (reverse search) in Elasticsearch

Introduction

Percolate queries help you find which stored queries match a new document. It is like searching backwards: instead of finding documents for a query, you find queries for a document.

You want to notify users when new content matches their saved searches.

You have many queries saved and want to quickly check which ones apply to a new event.

You want to filter incoming data by matching it against predefined conditions.

You want to build alert systems that trigger when certain data patterns appear.

You want to match documents against dynamic rules stored as queries.

Syntax

Elasticsearch

{
  "query": {
    "percolate": {
      "field": "query_field",
      "document": {
        "field1": "value1",
        "field2": "value2"
      }
    }
  }
}

The field is where your stored queries live in the index mapping.

The document is the new data you want to test against those queries.

Examples

This example checks which stored queries match a document with the message "Elasticsearch is cool".

Elasticsearch

{
  "query": {
    "percolate": {
      "field": "query",
      "document": {
        "message": "Elasticsearch is cool"
      }
    }
  }
}

This example tests stored queries against a document describing a user login event.

Elasticsearch

{
  "query": {
    "percolate": {
      "field": "query",
      "document": {
        "user": "alice",
        "action": "login"
      }
    }
  }
}

Sample Program

This program creates an index with a percolator field, stores a query that matches documents containing "error", then percolates a new document with a message containing "error" to find matching queries.

Elasticsearch

PUT /my-index
{
  "mappings": {
    "properties": {
      "query": {
        "type": "percolator"
      },
      "message": {
        "type": "text"
      }
    }
  }
}

# Index a stored query
PUT /my-index/_doc/1
{
  "query": {
    "match": {
      "message": "error"
    }
  }
}

# Percolate a new document to find matching queries
GET /my-index/_search
{
  "query": {
    "percolate": {
      "field": "query",
      "document": {
        "message": "There was an error in the system"
      }
    }
  }
}

Important Notes

Make sure your index mapping includes a percolator type field to store queries.

Percolate queries are useful for alerting and matching new data against saved conditions.

Stored queries must be indexed before you can percolate documents against them.

Summary

Percolate queries let you find stored queries that match a new document.

You need a special percolator field in your index mapping to store queries.

Use percolate queries to build alerting and reverse search features.