Scroll API for deep pagination in Elasticsearch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Scroll API request?

Given the following Elasticsearch Scroll API request snippet, what will be the value of hits.total.value in the first scroll response?

Elasticsearch

{
  "size": 2,
  "query": { "match_all": {} }
}

POST /my_index/_search?scroll=1m

Response snippet:
{
  "_scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA...",
  "hits": {
    "total": {"value": 5, "relation": "eq"},
    "hits": [
      {"_id": "1", "_source": {"field": "value1"}},
      {"_id": "2", "_source": {"field": "value2"}}
    ]
  }
}

A 5

B 2

C 0

D undefined

💡 Hint

The total number of matching documents is returned in hits.total.value, not the number of hits in the current batch.

The hits.total.value field shows the total number of documents matching the query, which is 5 here. The hits.hits array contains only the current batch of 2 documents.

🧠 Conceptual

intermediate

Why use Scroll API instead of from/size for deep pagination?

Which of the following is the main reason to use the Scroll API for deep pagination in Elasticsearch instead of using from and size parameters?

A Scroll API allows updating documents during pagination.

B Scroll API automatically sorts results by relevance score.

C Scroll API is faster and more efficient for retrieving large numbers of documents sequentially.

D Scroll API caches all results in memory permanently.

💡 Hint

Think about performance when retrieving many documents beyond the first few pages.

The Scroll API is designed to efficiently retrieve large sets of documents by keeping a snapshot of the data and returning batches sequentially. Using from and size for deep pagination is inefficient because Elasticsearch must skip many documents each time.

🔧 Debug

advanced

Identify the error in this Scroll API usage

What error will occur when running the following sequence of Elasticsearch Scroll API calls?

Elasticsearch

POST /my_index/_search?scroll=1m
{
  "size": 3,
  "query": { "match_all": {} }
}

POST /_search/scroll
{
  "scroll": "1m",
  "scroll_id": "incorrect_scroll_id"
}

A 404 Not Found error because the scroll ID is invalid or expired.

B No error; the scroll returns the next batch of results.

C Timeout error because the scroll duration is too short.

D SyntaxError due to malformed JSON in the scroll request.

💡 Hint

Consider what happens if you provide a wrong or expired scroll ID.

If the scroll ID is invalid or expired, Elasticsearch returns a 404 error indicating the scroll context was not found. This is common if the scroll ID is mistyped or the scroll timeout expired.

📝 Syntax

advanced

Which Scroll API request syntax is correct?

Choose the correct syntax for requesting the next batch of results using the Scroll API.

A

POST /my_index/_search/scroll
{
  "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA..."
}

B

POST /_search/scroll
{
  "scroll": "2m",
  "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA..."
}

C GET /_search/scroll?scroll=2m&scroll_id=DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA...

D

POST /_search/scroll
{
  "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA...",
  "timeout": "2m"
}

💡 Hint

Check the official Scroll API syntax for the request body and HTTP method.

The correct syntax uses a POST request to /_search/scroll with a JSON body containing scroll (the keep-alive time) and scroll_id. Option B uses GET which is invalid. Option B uses an incorrect endpoint. Option B uses an invalid parameter timeout instead of scroll.

🚀 Application

expert

How many documents are retrieved after 3 scroll requests?

You run a Scroll API search with size set to 4 on an index with 10 matching documents. You perform 3 scroll requests (initial search + 2 scrolls). How many documents have you retrieved in total?

A 6

B 8

C 12

D 10

💡 Hint

Multiply the batch size by the number of scroll requests, but consider the total documents available.

The initial search returns the first 4 documents, the first scroll returns the next 4, and the second scroll returns the remaining 2 documents (10 total). Total retrieved after 3 requests: 10 documents.

Practice

1. What is the main purpose of the Scroll API in Elasticsearch?

easy

A. To retrieve large sets of search results in small, manageable batches.

B. To update documents in bulk efficiently.

C. To delete old indices automatically.

D. To create new indices with custom mappings.

Solution

1. Step 1: Understand Scroll API usage

   The Scroll API is designed to handle large result sets by breaking them into smaller parts.

2. Step 2: Compare options with Scroll API purpose

   Options B, C, and D relate to other Elasticsearch features, not scrolling.

3. Final Answer:

   To retrieve large sets of search results in small, manageable batches. -> Option A

4. Quick Check:

   Scroll API = batch retrieval [OK]

Hint: Scroll API = fetch big results in small parts [OK]

Common Mistakes:

• Confusing Scroll API with bulk update operations
• Thinking Scroll API deletes or creates indices
• Assuming Scroll API returns all results at once

2. Which of the following is the correct way to start a scroll search request in Elasticsearch using JSON?

easy

A. {"scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA", "size": 100}

B. {"query": {"match_all": {}}, "scroll": "1m", "size": 100}

C. {"query": {"match": {"field": "value"}}, "timeout": "1m"}

D. {"scroll": "1m", "update": true}

Solution

1. Step 1: Identify scroll search syntax

   Starting a scroll requires a query, a scroll time, and size for batch size.

2. Step 2: Analyze options

   {"query": {"match_all": {}}, "scroll": "1m", "size": 100} includes query, scroll time, and size correctly. {"scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA", "size": 100} uses scroll_id which is for continuing scroll, not starting. {"query": {"match": {"field": "value"}}, "timeout": "1m"} lacks scroll parameter. {"scroll": "1m", "update": true} has invalid update field.

3. Final Answer:

   {"query": {"match_all": {}}, "scroll": "1m", "size": 100} -> Option B

4. Quick Check:

   Start scroll = query + scroll + size [OK]

Hint: Start scroll with query + scroll + size keys [OK]

Common Mistakes:

• Using scroll_id to start scroll instead of continue
• Omitting the scroll parameter
• Confusing scroll with timeout or update

3. Given the following scroll response snippet, what is the correct next step to fetch more results?

{
  "_scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAA",
  "hits": {"hits": [{"_id": "1"}, {"_id": "2"}]}
}

medium

A. Send a new search request without scroll_id.

B. Delete the scroll_id to reset the scroll context.

C. Use the scroll_id in a subsequent scroll request with the scroll parameter.

D. Use the hits array to manually fetch documents by ID.

Solution

1. Step 1: Understand scroll continuation

   To get next batch, use the scroll_id from previous response with scroll parameter.

2. Step 2: Evaluate options

   Use the scroll_id in a subsequent scroll request with the scroll parameter. correctly describes using scroll_id and scroll to continue. Send a new search request without scroll_id. restarts search, losing context. Delete the scroll_id to reset the scroll context. is incorrect as deleting scroll_id is not valid. Use the hits array to manually fetch documents by ID. is manual and inefficient.

3. Final Answer:

   Use the scroll_id in a subsequent scroll request with the scroll parameter. -> Option C

4. Quick Check:

   Next scroll = scroll_id + scroll [OK]

Hint: Use scroll_id + scroll param to get next batch [OK]

Common Mistakes:

• Restarting search instead of continuing scroll
• Ignoring scroll parameter in next request
• Trying to fetch documents manually by ID

4. You wrote this scroll request but get an error: {"scroll_id": "abc123"}. What is the likely cause?

medium

A. Missing the scroll parameter to keep the scroll context alive.

B. The scroll_id is invalid and must be a number.

C. You cannot use scroll_id in a scroll request.

D. The size parameter is required with scroll_id.

Solution

1. Step 1: Check scroll request requirements

   When continuing a scroll, the scroll parameter (time) must be included to keep context alive.

2. Step 2: Analyze error cause

   Missing the scroll parameter to keep the scroll context alive. correctly identifies missing scroll parameter. The scroll_id is invalid and must be a number. is wrong; scroll_id is a string. You cannot use scroll_id in a scroll request. is false; scroll_id is needed. The size parameter is required with scroll_id. is incorrect; size is not required in scroll continuation.

3. Final Answer:

   Missing the scroll parameter to keep the scroll context alive. -> Option A

4. Quick Check:

   Scroll continuation needs scroll param [OK]

Hint: Always include scroll param with scroll_id [OK]

Common Mistakes:

• Omitting scroll parameter in scroll continuation
• Assuming scroll_id must be numeric
• Thinking size is needed every scroll request

5. You want to retrieve 10,000 documents using the Scroll API. Which approach is best to avoid memory issues and ensure all documents are retrieved?

hard

A. Use the Scroll API but do not specify the scroll parameter to speed up retrieval.

B. Set size to 10,000 in a single search request without scrolling.

C. Fetch documents by IDs one by one using separate queries.

D. Use a scroll time of 1 minute and fetch batches of 100 documents repeatedly until no hits remain.

Solution

1. Step 1: Understand deep pagination with Scroll API

   Scroll API is designed to fetch large results in small batches with a scroll timeout to keep context alive.

2. Step 2: Evaluate options for best practice

   Use a scroll time of 1 minute and fetch batches of 100 documents repeatedly until no hits remain. correctly uses scroll time and batch size to safely retrieve all documents. Set size to 10,000 in a single search request without scrolling. risks memory overload. Use the Scroll API but do not specify the scroll parameter to speed up retrieval. is invalid because scroll param is required. Fetch documents by IDs one by one using separate queries. is inefficient and slow.

3. Final Answer:

   Use a scroll time of 1 minute and fetch batches of 100 documents repeatedly until no hits remain. -> Option D

4. Quick Check:

   Scroll API + batch + scroll time = safe deep pagination [OK]

Hint: Fetch in batches with scroll time to avoid overload [OK]

Common Mistakes:

• Requesting all documents at once causing memory errors
• Omitting scroll parameter to speed up
• Fetching documents individually instead of batches