Cross-cluster search in Elasticsearch - Mini Project: Build & Apply

Cross-cluster Search with Elasticsearch

📖 Scenario: You work in a company that has two Elasticsearch clusters: clusterA and clusterB. Each cluster stores product data for different regions. You want to search products across both clusters in one query.

🎯 Goal: Build a cross-cluster search query that searches the products index on both clusterA and clusterB clusters for products with the word "laptop" in their name field.

📋 What You'll Learn

Create a search query targeting the products index on clusterA and clusterB using cross-cluster search syntax.

Add a match query to find documents where the name field contains the word "laptop".

Limit the search results to 5 documents.

Print the final JSON query that can be sent to Elasticsearch.

💡 Why This Matters

🌍 Real World

Companies with data spread across multiple Elasticsearch clusters can search all data in one query using cross-cluster search.

💼 Career

Knowing cross-cluster search helps you build scalable search solutions and work with distributed Elasticsearch setups common in large organizations.

1

Setup the cross-cluster search index pattern

Create a variable called index_pattern and set it to the string "clusterA:products,clusterB:products" which specifies the indices to search across clusters.

Elasticsearch

# Create a variable called index_pattern with the value "clusterA:products,clusterB:products"
# Your code here

Hint

Use the format cluster_name:index_name separated by commas for multiple clusters.

2

Create the match query for the product name

Create a variable called query_body and set it to a dictionary with a query key. The value should be a match query that searches for the word "laptop" in the name field.

Elasticsearch

index_pattern = "clusterA:products,clusterB:products"
# Create a variable called query_body with a match query for name containing "laptop"
# Your code here

Hint

Use the match query inside the query key to find the word "laptop" in the name field.

3

Add a size limit to the query

Add a size key to the query_body dictionary and set its value to 5 to limit the number of results returned.

Elasticsearch

index_pattern = "clusterA:products,clusterB:products"
query_body = {
    "query": {
        "match": {
            "name": "laptop"
        }
    }
}
# Add size: 5 to query_body
# Your code here

Hint

The size key controls how many results Elasticsearch returns.

4

Print the final search query

Print the index_pattern and the query_body variables to show the full cross-cluster search query.

Elasticsearch

index_pattern = "clusterA:products,clusterB:products"
query_body = {
    "query": {
        "match": {
            "name": "laptop"
        }
    },
    "size": 5
}
# Print index_pattern and query_body
# Your code here

Hint

Use print() to display both variables clearly.

Practice

1. What is the main purpose of cross-cluster search in Elasticsearch?

easy

A. To monitor cluster health status remotely

B. To backup data from one cluster to another

C. To merge two clusters into one

D. To search data across multiple Elasticsearch clusters using a single query

Solution

1. Step 1: Understand cross-cluster search concept

   Cross-cluster search allows querying data from multiple clusters in one search request.

2. Step 2: Differentiate from other cluster operations

   It does not merge clusters, backup data, or monitor health but focuses on searching data.

3. Final Answer:

   To search data across multiple Elasticsearch clusters using a single query -> Option D

4. Quick Check:

   Cross-cluster search = search across clusters [OK]

Hint: Cross-cluster search = one query, many clusters [OK]

Common Mistakes:

• Confusing search with backup or monitoring
• Thinking it merges clusters
• Assuming it manages cluster health

2. Which syntax correctly specifies a remote cluster alias in a cross-cluster search query?

easy

A. GET /remote_cluster:index/_search

B. GET /index@remote_cluster/_search

C. GET /index/remote_cluster/_search

D. GET /remote_cluster/_search/index

Solution

1. Step 1: Recall remote cluster alias syntax

   The correct syntax uses remote_cluster:index to specify the cluster alias and index.

2. Step 2: Check each option format

   Only GET /remote_cluster:index/_search matches the correct pattern: GET /remote_cluster:index/_search.

3. Final Answer:

   GET /remote_cluster:index/_search -> Option A

4. Quick Check:

   Alias:index/_search = correct syntax [OK]

Hint: Use alias:index/_search to target remote cluster data [OK]

Common Mistakes:

• Placing alias after index
• Using slashes instead of colon
• Misordering parts of the URL

3. Given this cross-cluster search query:

GET /clusterA:logs-2023/_search
{
  "query": { "match_all": {} }
}

What data will this query return?

medium

A. All documents from the local cluster's logs-2023 index

B. All documents from the logs-2023 index in clusterA

C. Documents matching "clusterA" in the logs-2023 index

D. An error because cluster alias is missing

Solution

1. Step 1: Identify cluster alias usage

   The query uses clusterA:logs-2023, meaning it targets the logs-2023 index on remote cluster named clusterA.

2. Step 2: Understand the query body

   The match_all query returns all documents from that index on clusterA.

3. Final Answer:

   All documents from the logs-2023 index in clusterA -> Option B

4. Quick Check:

   Alias:index with match_all = all remote docs [OK]

Hint: Alias:index means search that index on remote cluster [OK]

Common Mistakes:

• Assuming it searches local cluster
• Thinking it filters by cluster name in data
• Believing alias is optional

4. You run this cross-cluster search query:

GET /remoteCluster:products/_search
{
  "query": { "term": { "category": "electronics" } }
}

But get an error: no such remote cluster. What is the likely cause?

medium

A. The query syntax is invalid for cross-cluster search

B. The index 'products' does not exist on the remote cluster

C. The remote cluster alias 'remoteCluster' is not configured in the local cluster

D. The term query cannot be used in cross-cluster search

Solution

1. Step 1: Analyze the error message

   The error no such remote cluster means the alias 'remoteCluster' is unknown to the local cluster.

2. Step 2: Check configuration requirements

   Remote clusters must be configured before use; missing alias causes this error.

3. Final Answer:

   The remote cluster alias 'remoteCluster' is not configured in the local cluster -> Option C

4. Quick Check:

   Missing alias config = no such remote cluster error [OK]

Hint: Configure remote cluster alias before querying [OK]

Common Mistakes:

• Assuming index absence causes this error
• Blaming query syntax for alias errors
• Thinking term queries are unsupported

5. You want to search the sales-2023 index across two remote clusters named clusterX and clusterY. Which query correctly searches both clusters and returns combined results?

hard

A. GET /clusterX:sales-2023,clusterY:sales-2023/_search
{ "query": { "match_all": {} } }

B. GET /sales-2023/_search
{ "query": { "match_all": {} }, "clusters": ["clusterX", "clusterY"] }

C. GET /clusterX:clusterY:sales-2023/_search
{ "query": { "match_all": {} } }

D. GET /sales-2023/_search
{ "query": { "match_all": {} }, "remote_clusters": ["clusterX", "clusterY"] }

Solution

1. Step 1: Recall syntax for multiple remote clusters

   To search multiple clusters, use comma-separated list of <code>cluster_alias:index</code>, like <code>clusterX:sales-2023,clusterY:sales-2023</code>.

2. Step 2: Evaluate each option

   GET /clusterX:sales-2023,clusterY:sales-2023/_search { "query": { "match_all": {} } } uses <code>clusterX:sales-2023,clusterY:sales-2023</code> which is correct syntax for cross-cluster search across multiple clusters.

3. Final Answer:

   GET /clusterX:sales-2023,clusterY:sales-2023/_search { "query": { "match_all": {} } } -> Option A

4. Quick Check:

   clusterX:sales-2023,clusterY:sales-2023/_search = multi-cluster search [OK]

Hint: comma-separate alias:index for multi-cluster search [OK]

Common Mistakes:

• Using multiple colons instead of commas
• Adding cluster names inside query body
• Assuming local index searches multiple clusters