Cross-cluster search in Elasticsearch

Introduction

Cross-cluster search lets you look for information in many Elasticsearch clusters at once. This helps you find data quickly without moving it all to one place.

You have data spread across different Elasticsearch clusters in different locations.

You want to search all your data without copying it to a single cluster.

You manage multiple teams or projects with separate clusters but need combined search results.

You want to keep data isolated for security but still allow searching across clusters.

Syntax

Elasticsearch

GET /_search
{
  "query": {
    "match": {
      "field": "value"
    }
  }
}

# To search across clusters, prefix index names with cluster alias:
GET /cluster_alias:index_name/_search
{
  "query": {
    "match_all": {}
  }
}

You must set up remote cluster connections in your Elasticsearch settings before using cross-cluster search.

Use cluster_alias:index_name to specify which cluster and index to search.

Examples

This searches the logs-2024 index on the remote cluster named remote_cluster_1 for documents containing "error" in the message field.

Elasticsearch

GET /remote_cluster_1:logs-2024/_search
{
  "query": {
    "match": {
      "message": "error"
    }
  }
}

This searches both the local index local_index and the events index on the remote cluster remote_cluster_2 and returns all documents.

Elasticsearch

GET /local_index,remote_cluster_2:events/_search
{
  "query": {
    "match_all": {}
  }
}

Sample Program

First, we tell Elasticsearch where to find the remote cluster by adding its address. Then, we search the products index on that remote cluster for documents where the name contains "laptop".

Elasticsearch

PUT /_cluster/settings
{
  "persistent": {
    "search.remote.remote_cluster_1.seeds": ["192.168.1.10:9300"]
  }
}

GET /remote_cluster_1:products/_search
{
  "query": {
    "match": {
      "name": "laptop"
    }
  }
}

Important Notes

Make sure the remote cluster is reachable and configured correctly before searching.

Cross-cluster search can add some delay because it talks to multiple clusters.

You can combine local and remote indices in one search request.

Summary

Cross-cluster search helps you find data across many Elasticsearch clusters easily.

You must set up remote cluster connections before using it.

Use cluster aliases to specify which cluster's data you want to search.

Practice

1. What is the main purpose of cross-cluster search in Elasticsearch?

easy

A. To monitor cluster health status remotely

B. To backup data from one cluster to another

C. To merge two clusters into one

D. To search data across multiple Elasticsearch clusters using a single query

Solution

1. Step 1: Understand cross-cluster search concept

   Cross-cluster search allows querying data from multiple clusters in one search request.

2. Step 2: Differentiate from other cluster operations

   It does not merge clusters, backup data, or monitor health but focuses on searching data.

3. Final Answer:

   To search data across multiple Elasticsearch clusters using a single query -> Option D

4. Quick Check:

   Cross-cluster search = search across clusters [OK]

Hint: Cross-cluster search = one query, many clusters [OK]

Common Mistakes:

• Confusing search with backup or monitoring
• Thinking it merges clusters
• Assuming it manages cluster health

2. Which syntax correctly specifies a remote cluster alias in a cross-cluster search query?

easy

A. GET /remote_cluster:index/_search

B. GET /index@remote_cluster/_search

C. GET /index/remote_cluster/_search

D. GET /remote_cluster/_search/index

Solution

1. Step 1: Recall remote cluster alias syntax

   The correct syntax uses remote_cluster:index to specify the cluster alias and index.

2. Step 2: Check each option format

   Only GET /remote_cluster:index/_search matches the correct pattern: GET /remote_cluster:index/_search.

3. Final Answer:

   GET /remote_cluster:index/_search -> Option A

4. Quick Check:

   Alias:index/_search = correct syntax [OK]

Hint: Use alias:index/_search to target remote cluster data [OK]

Common Mistakes:

• Placing alias after index
• Using slashes instead of colon
• Misordering parts of the URL

3. Given this cross-cluster search query:

GET /clusterA:logs-2023/_search
{
  "query": { "match_all": {} }
}

What data will this query return?

medium

A. All documents from the local cluster's logs-2023 index

B. All documents from the logs-2023 index in clusterA

C. Documents matching "clusterA" in the logs-2023 index

D. An error because cluster alias is missing

Solution

1. Step 1: Identify cluster alias usage

   The query uses clusterA:logs-2023, meaning it targets the logs-2023 index on remote cluster named clusterA.

2. Step 2: Understand the query body

   The match_all query returns all documents from that index on clusterA.

3. Final Answer:

   All documents from the logs-2023 index in clusterA -> Option B

4. Quick Check:

   Alias:index with match_all = all remote docs [OK]

Hint: Alias:index means search that index on remote cluster [OK]

Common Mistakes:

• Assuming it searches local cluster
• Thinking it filters by cluster name in data
• Believing alias is optional

4. You run this cross-cluster search query:

GET /remoteCluster:products/_search
{
  "query": { "term": { "category": "electronics" } }
}

But get an error: no such remote cluster. What is the likely cause?

medium

A. The query syntax is invalid for cross-cluster search

B. The index 'products' does not exist on the remote cluster

C. The remote cluster alias 'remoteCluster' is not configured in the local cluster

D. The term query cannot be used in cross-cluster search

Solution

1. Step 1: Analyze the error message

   The error no such remote cluster means the alias 'remoteCluster' is unknown to the local cluster.

2. Step 2: Check configuration requirements

   Remote clusters must be configured before use; missing alias causes this error.

3. Final Answer:

   The remote cluster alias 'remoteCluster' is not configured in the local cluster -> Option C

4. Quick Check:

   Missing alias config = no such remote cluster error [OK]

Hint: Configure remote cluster alias before querying [OK]

Common Mistakes:

• Assuming index absence causes this error
• Blaming query syntax for alias errors
• Thinking term queries are unsupported

5. You want to search the sales-2023 index across two remote clusters named clusterX and clusterY. Which query correctly searches both clusters and returns combined results?

hard

A. GET /clusterX:sales-2023,clusterY:sales-2023/_search
{ "query": { "match_all": {} } }

B. GET /sales-2023/_search
{ "query": { "match_all": {} }, "clusters": ["clusterX", "clusterY"] }

C. GET /clusterX:clusterY:sales-2023/_search
{ "query": { "match_all": {} } }

D. GET /sales-2023/_search
{ "query": { "match_all": {} }, "remote_clusters": ["clusterX", "clusterY"] }

Solution

1. Step 1: Recall syntax for multiple remote clusters

   To search multiple clusters, use comma-separated list of <code>cluster_alias:index</code>, like <code>clusterX:sales-2023,clusterY:sales-2023</code>.

2. Step 2: Evaluate each option

   GET /clusterX:sales-2023,clusterY:sales-2023/_search { "query": { "match_all": {} } } uses <code>clusterX:sales-2023,clusterY:sales-2023</code> which is correct syntax for cross-cluster search across multiple clusters.

3. Final Answer:

   GET /clusterX:sales-2023,clusterY:sales-2023/_search { "query": { "match_all": {} } } -> Option A

4. Quick Check:

   clusterX:sales-2023,clusterY:sales-2023/_search = multi-cluster search [OK]

Hint: comma-separate alias:index for multi-cluster search [OK]

Common Mistakes:

• Using multiple colons instead of commas
• Adding cluster names inside query body
• Assuming local index searches multiple clusters