Why Async search for expensive queries in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if your search could run quietly in the background while you keep working?

The Scenario

Imagine you have a huge library of books and you want to find all books about a rare topic. You ask your friend to look through every shelf, but it takes a very long time. Meanwhile, you have to wait and can't do anything else.

The Problem

Doing this search manually means you wait a long time for the answer. If the search is very slow, your app or website might freeze or become unresponsive. Also, if the search fails, you lose all progress and must start over.

The Solution

Async search lets you start the search and then check back later for the results. This way, your app stays fast and responsive. You can do other things while waiting, and even handle very big searches without freezing.

Before vs After

✗ Before

POST /_search
{
  "query": { "match": { "text": "rare topic" } }
}

✓ After

POST /_async_search
{
  "query": { "match": { "text": "rare topic" } }
}

What It Enables

Async search makes it easy to handle slow, heavy searches without blocking your users or systems.

Real Life Example

A news website uses async search to find all articles about a breaking event across millions of records, letting readers keep browsing while the search runs.

Key Takeaways

Manual searches can block and slow down apps.

Async search runs queries in the background.

This keeps apps responsive and handles big data smoothly.

Practice

1. What is the main benefit of using async search in Elasticsearch for expensive queries?

easy

A. It caches all query results permanently.

B. It automatically speeds up the query execution time.

C. It disables query logging to improve performance.

D. It allows running slow queries without blocking the application.

Solution

1. Step 1: Understand async search purpose

   Async search lets you run slow or heavy queries without making your app wait or freeze.

2. Step 2: Identify the main benefit

   This means your app can continue working while the query runs in the background.

3. Final Answer:

   It allows running slow queries without blocking the application. -> Option D

4. Quick Check:

   Async search = non-blocking query execution [OK]

Hint: Async search runs queries in background, so app doesn't wait [OK]

Common Mistakes:

• Thinking async search speeds up queries automatically
• Assuming async search caches results permanently
• Believing async search disables logging

2. Which of the following is the correct way to start an async search request in Elasticsearch using the REST API?

easy

A. POST /_async_search
{
"query": { "match_all": {} }
}

B. GET /_async_search
{
"query": { "match_all": {} }
}

C. POST /_search/async
{
"query": { "match_all": {} }
}

D. PUT /_async_search
{
"query": { "match_all": {} }
}

Solution

1. Step 1: Recall async search API endpoint

   The correct endpoint to start an async search is POST /_async_search with the query in the body.

2. Step 2: Check HTTP method and path

   GET is not used to start async search, and /_search/async or PUT are incorrect paths or methods.

3. Final Answer:

   POST /_async_search with query body -> Option A

4. Quick Check:

   Start async search = POST /_async_search [OK]

Hint: Use POST method on /_async_search to start async search [OK]

Common Mistakes:

• Using GET instead of POST to start async search
• Using wrong endpoint like /_search/async
• Using PUT method which is invalid here

3. Given this async search response snippet, what does the id field represent?

{
  "id": "r1A2B3C4D5E6F7G8H9I",
  "is_running": true,
  "response": null
}

medium

A. The timeout duration for the async search.

B. The total number of documents matched by the query.

C. The unique identifier to check status or fetch results later.

D. The Elasticsearch node handling the query.

Solution

1. Step 1: Understand the async search response fields

   The id is a unique string to identify this async search request.

2. Step 2: Purpose of the id

   You use this id to check if the search is done or to get the results later.

3. Final Answer:

   The unique identifier to check status or fetch results later. -> Option C

4. Quick Check:

   Async search id = unique query handle [OK]

Hint: Async search id tracks query status and results [OK]

Common Mistakes:

• Confusing id with document count
• Thinking id is timeout or node info
• Assuming id changes during query

4. You wrote this code to start an async search but get an error:

POST /_async_search
{
  "query": {
    "match": {
      "title": "Elasticsearch"
    }
  },
  "wait_for_completion_timeout": "1s"
}

What is the error in this request?

medium

A. Missing comma between query and wait_for_completion_timeout fields.

B. Using POST instead of GET method.

C. wait_for_completion_timeout cannot be set in the request body.

D. The field name "title" is invalid in match query.

Solution

1. Step 1: Check JSON syntax

   The JSON body is missing a comma after the closing brace of the "query" object.

2. Step 2: Validate method and fields

   POST is correct method, wait_for_completion_timeout is valid in body, and "title" is a valid field name.

3. Final Answer:

   Missing comma between query and wait_for_completion_timeout fields. -> Option A

4. Quick Check:

   JSON syntax error = missing comma [OK]

Hint: Check commas between JSON fields carefully [OK]

Common Mistakes:

• Forgetting commas between JSON objects
• Confusing HTTP methods for async search
• Misplacing wait_for_completion_timeout outside body

5. You want to run a very expensive aggregation query on a large dataset without timing out. Which approach using async search is best to get the final results efficiently?

hard

A. Run a normal search with a very high timeout value to wait for results.

B. Start async search with a long wait_for_completion_timeout and poll using the returned id until results are ready.

C. Start async search and immediately request results without waiting for completion.

D. Run the query multiple times with smaller timeouts and merge results manually.

Solution

1. Step 1: Understand async search timeout and polling

   Setting a reasonable wait_for_completion_timeout lets the server try to finish quickly but returns control if it takes longer.

2. Step 2: Use the returned id to poll for completion

   You can check the status later using the id until the results are ready, avoiding timeouts and blocking.

3. Final Answer:

   Start async search with a long wait_for_completion_timeout and poll using the returned id until results are ready. -> Option B

4. Quick Check:

   Async search + polling = efficient for expensive queries [OK]

Hint: Use wait_for_completion_timeout + poll with id for big queries [OK]

Common Mistakes:

• Using normal search with high timeout risking app freeze
• Requesting results immediately before completion
• Manually merging partial results instead of async search