Async search for expensive queries in Elasticsearch - Mini Project: Build & Apply

Async search for expensive queries

📖 Scenario: You work with a large Elasticsearch database that stores product sales data. Some queries take a long time to run because they analyze a lot of data. To avoid waiting and blocking your application, you want to use Elasticsearch's async search feature. This lets you start a search and check back later for the results.

🎯 Goal: Build a simple async search workflow using Elasticsearch's REST API. You will start an async search for products with sales over a certain amount, then check the status and finally get the results.

📋 What You'll Learn

Create an async search request with a query for products with sales greater than 1000

Store the async search ID returned by Elasticsearch

Use the async search ID to check the status of the search

Retrieve and print the final search results

💡 Why This Matters

🌍 Real World

Async search is useful when queries take a long time on large datasets. It lets applications stay responsive by checking back later for results.

💼 Career

Many data engineer and backend developer roles require working with Elasticsearch and optimizing search queries using async search for better performance.

1

Create async search request

Write a POST request to /products/_async_search with a JSON body that queries for products where sales is greater than 1000. Store the response JSON in a variable called response. Use the Elasticsearch Python client method client.async_search.submit with index "products" and the query body shown below.

Elasticsearch

# Your code here to submit async search

Need a hint?

Use client.async_search.submit with the correct index and query body to start the async search.

2

Store async search ID

Extract the async search ID from response and store it in a variable called search_id. The ID is in response['id'].

Elasticsearch

from elasticsearch import Elasticsearch

client = Elasticsearch()

query_body = {
    "query": {
        "range": {
            "sales": {
                "gt": 1000
            }
        }
    }
}

response = client.async_search.submit(index="products", body=query_body)

# Extract async search ID from response
# Your code here

Need a hint?

The async search ID is returned in the 'id' field of the response.

3

Check async search status

Use the search_id to get the current status of the async search by calling client.async_search.get with id=search_id. Store the result in a variable called status_response.

Elasticsearch

from elasticsearch import Elasticsearch

client = Elasticsearch()

query_body = {
    "query": {
        "range": {
            "sales": {
                "gt": 1000
            }
        }
    }
}

response = client.async_search.submit(index="products", body=query_body)

search_id = response['id']

# Get async search status
# Your code here

Need a hint?

Use client.async_search.get with the async search ID to check the status.

4

Print async search results

Print the hits from the async search results stored in status_response. The hits are in status_response['response']['hits']['hits']. Use print() to display them.

Elasticsearch

from elasticsearch import Elasticsearch

client = Elasticsearch()

query_body = {
    "query": {
        "range": {
            "sales": {
                "gt": 1000
            }
        }
    }
}

response = client.async_search.submit(index="products", body=query_body)

search_id = response['id']

status_response = client.async_search.get(id=search_id)

# Print the search hits
# Your code here

Need a hint?

Print the list of hits from status_response['response']['hits']['hits']. It may be empty if no products match.