Search performance tuning in Elasticsearch - Mini Project: Build & Apply

Search Performance Tuning in Elasticsearch

📖 Scenario: You are working on a website that uses Elasticsearch to help users find products quickly. Sometimes, the search results are slow because the queries are not optimized. You want to improve the speed of search queries by tuning the Elasticsearch settings and queries.

🎯 Goal: Build a simple Elasticsearch query and tune it step-by-step to improve search performance by using filters, limiting fields, and sorting efficiently.

📋 What You'll Learn

Create an Elasticsearch index with sample product data

Add a filter to the search query to reduce the search scope

Limit the fields returned by the query to only necessary ones

Sort the search results efficiently and print the final query

💡 Why This Matters

🌍 Real World

Optimizing search queries in Elasticsearch helps websites and apps deliver faster and more relevant search results to users, improving user experience.

💼 Career

Many jobs in data engineering, backend development, and DevOps require knowledge of Elasticsearch tuning to handle large-scale search systems efficiently.

1

Create the Elasticsearch index with sample data

Create an Elasticsearch index called products with the following sample documents: {"id": 1, "name": "Red Shirt", "category": "clothing", "price": 29.99}, {"id": 2, "name": "Blue Jeans", "category": "clothing", "price": 49.99}, and {"id": 3, "name": "Coffee Mug", "category": "kitchen", "price": 9.99}. Use the bulk API format for indexing.

Elasticsearch

# Your code here to create the index and add the sample documents using bulk API

Hint

Use the Elasticsearch bulk API format with { "index": { "_id": X } } lines followed by the document JSON.

2

Add a filter to the search query

Create a search query that filters products to only those in the clothing category using a term filter inside a bool query. Assign this query to a variable called search_query.

Elasticsearch

POST /products/_search
# Your code here to create search_query with a bool filter for category 'clothing'

Hint

Use a bool query with a filter clause containing a term query for the category.

3

Limit the fields returned by the query

Modify the search_query to include a _source field that only returns id, name, and price fields to reduce data size.

Elasticsearch

POST /products/_search
# Your code here to add _source filtering for id, name, and price

Hint

Add the _source key at the top level of the query JSON with a list of fields to return.

4

Sort the search results by price ascending

Add a sort clause to the search_query that sorts the results by price in ascending order. Then print the entire final query JSON.

Elasticsearch

POST /products/_search
# Your code here to add sorting by price ascending and print the final query

Hint

Add a sort key with a list containing a dictionary for price ascending order.

Practice

1. Which of the following is a common way to improve search performance in Elasticsearch?

easy

A. Limit the number of results returned using size parameter

B. Increase the number of shards without limit

C. Disable caching completely

D. Use wildcard queries on all fields

Solution

1. Step 1: Understand result limiting

   Limiting results with size reduces data processed and returned, speeding up queries.

2. Step 2: Evaluate other options

   Increasing shards without limit can hurt performance, disabling cache reduces speed, and wildcard queries are slow.

3. Final Answer:

   Limit the number of results returned using size parameter -> Option A

4. Quick Check:

   Limiting results = faster search [OK]

Hint: Use size to limit results for faster queries [OK]

Common Mistakes:

• Thinking more shards always improve speed
• Ignoring caching benefits
• Using wildcard queries on all fields

2. Which Elasticsearch query syntax correctly limits the returned fields to only title and author?

easy

A. {"return_fields": ["title", "author"], "query": {"match_all": {}}}

B. {"fields": ["title", "author"], "query": {"match_all": {}}}

C. {"select": ["title", "author"], "query": {"match_all": {}}}

D. {"_source": ["title", "author"], "query": {"match_all": {}}}

Solution

1. Step 1: Identify correct field limiting syntax

   Elasticsearch uses _source to specify which fields to return.

2. Step 2: Check other options

   fields, select, and return_fields are not valid for limiting returned fields in this context.

3. Final Answer:

   {"_source": ["title", "author"], "query": {"match_all": {}}} -> Option D

4. Quick Check:

   Use _source to limit fields [OK]

Hint: Use _source to specify returned fields [OK]

Common Mistakes:

• Using fields instead of _source
• Trying SQL-like select syntax
• Using unsupported keys like return_fields

3. Given this Elasticsearch query, what will be the effect of adding "timeout": "2s"?

{
  "query": {"match": {"content": "fast search"}},
  "timeout": "2s"
}

medium

A. The query will fail if it takes longer than 2 seconds

B. The query will cache results for 2 seconds

C. The query will return partial results after 2 seconds

D. The query will wait 2 seconds before starting

Solution

1. Step 1: Understand timeout behavior

   Elasticsearch's timeout stops the query after the specified time and returns partial results if available.

2. Step 2: Evaluate other options

   It does not fail immediately, does not delay start, and does not control caching.

3. Final Answer:

   The query will return partial results after 2 seconds -> Option C

4. Quick Check:

   timeout returns partial results [OK]

Hint: Timeout returns partial results if query is slow [OK]

Common Mistakes:

• Assuming timeout causes query failure
• Thinking timeout delays query start
• Confusing timeout with caching duration

4. You have this query to limit results and fields:

{
  "size": 10,
  "query": {
    "_source": ["title", "date"],
    "match_all": {}
  }
}

But the query returns all fields. What is the likely mistake?

medium

A. Using size instead of limit

B. Using _source inside the query body instead of top-level

C. Missing fields parameter to limit fields

D. The match_all query ignores field limits

Solution

1. Step 1: Check placement of _source

   _source must be at the top level of the query JSON, not inside query.

2. Step 2: Review other options

   fields is deprecated for this purpose, size is correct, and match_all does not ignore field limits.

3. Final Answer:

   Using _source inside the query body instead of top-level -> Option B

4. Quick Check:

   _source must be top-level [OK]

Hint: Place _source at top level, not inside query [OK]

Common Mistakes:

• Putting _source inside query
• Confusing size with limit
• Assuming match_all ignores field filtering

5. You want to optimize a search that returns many documents but only needs the id and summary fields, and must respond within 1 second. Which combination of settings best improves performance?

hard

A. Set size to a low number, use _source to limit fields, and add timeout of 1s

B. Set size high, disable _source, and remove timeout

C. Use wildcard queries on all fields and set timeout to 5s

D. Increase shards count and use fields to limit fields

Solution

1. Step 1: Limit results and fields

   Setting size low reduces returned documents; _source limits fields to needed ones.

2. Step 2: Use timeout to keep response fast

   Adding timeout of 1 second ensures query won't hang and keeps system responsive.

3. Step 3: Evaluate other options

   High size and disabling _source increase load; wildcard queries are slow; increasing shards without need can hurt performance.

4. Final Answer:

   Set size to a low number, use _source to limit fields, and add timeout of 1s -> Option A

5. Quick Check:

   Limit size + fields + timeout = best performance [OK]

Hint: Limit size, fields, and add timeout for fast, efficient search [OK]

Common Mistakes:

• Setting size too high
• Disabling field filtering
• Ignoring timeout setting
• Increasing shards unnecessarily