Why Source filtering in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if you could instantly get only the exact data you want from thousands of documents?

The Scenario

Imagine you have a huge box filled with thousands of documents, but you only want to see a few specific details from each one. Without a way to pick just what you need, you have to open every document and sift through all the information manually.

The Problem

Manually searching through every document is slow and tiring. It wastes time and computer power because you get overwhelmed with too much unnecessary data. This can cause delays and mistakes when trying to find the important parts.

The Solution

Source filtering lets you tell Elasticsearch exactly which parts of the documents you want to see. It quickly skips all the extra data and only shows the fields you care about, making your searches faster and cleaner.

Before vs After

✗ Before

{ "query": { "match_all": {} } }

✓ After

{ "_source": ["title", "author"], "query": { "match_all": {} } }

What It Enables

Source filtering makes it easy to get just the useful information you need, saving time and making your data clearer and faster to work with.

Real Life Example

When searching a library catalog, you might only want to see the book titles and authors, not the full descriptions or reviews. Source filtering helps you get just those details instantly.

Key Takeaways

Manually handling full documents is slow and overwhelming.

Source filtering lets you pick only the fields you want.

This speeds up searches and simplifies your results.

Practice

1. What is the main purpose of _source filtering in Elasticsearch?

easy

A. To control which parts of the document are returned in search results

B. To filter documents before indexing

C. To change the document content during indexing

D. To sort search results by field values

Solution

1. Step 1: Understand _source filtering role

   _source filtering controls which fields of a document are included in the search response.

2. Step 2: Differentiate from other features

   It does not affect indexing or sorting, only what data is returned after a search.

3. Final Answer:

   To control which parts of the document are returned in search results -> Option A

4. Quick Check:

   Source filtering = control returned fields [OK]

Hint: Remember: source filtering limits returned fields only [OK]

Common Mistakes:

• Confusing source filtering with indexing filters
• Thinking it changes document content
• Assuming it sorts results

2. Which of the following is the correct syntax to exclude the field password from the search results using source filtering?

easy

A. "_source": { "filter": ["password"] }

B. "_source": { "include": ["password"] }

C. "_source": { "excludes": ["password"] }

D. "_source": { "remove": ["password"] }

Solution

1. Step 1: Recall source filtering syntax

   To exclude fields, use "excludes" inside _source with a list of field names.

2. Step 2: Match correct syntax

   "_source": { "excludes": ["password"] } uses "excludes": ["password"], which is the correct way to exclude the password field.

3. Final Answer:

   "_source": { "excludes": ["password"] } -> Option C

4. Quick Check:

   Exclude fields = use "excludes" key [OK]

Hint: Use "excludes" key to remove fields from results [OK]

Common Mistakes:

• Using "include" to exclude fields
• Using non-existent keys like "filter" or "remove"
• Wrong JSON structure

3. Given this Elasticsearch query snippet, what fields will be returned in the search results?

{
  "_source": { "includes": ["title", "author"] },
  "query": { "match_all": {} }
}

medium

A. All fields will be returned

B. All fields except title and author will be returned

C. No fields will be returned

D. Only the fields title and author will be returned

Solution

1. Step 1: Understand includes in _source

   The includes key specifies which fields to return from the document.

2. Step 2: Apply to given query

   Since only title and author are included, only these fields will appear in the results.

3. Final Answer:

   Only the fields title and author will be returned -> Option D

4. Quick Check:

   includes fields = return only those fields [OK]

Hint: includes means return only listed fields [OK]

Common Mistakes:

• Thinking includes excludes fields
• Assuming all fields return by default
• Confusing includes with excludes

4. Identify the error in this source filtering snippet:

{
  "_source": {
    "includes": "title,author"
  },
  "query": { "match_all": {} }
}

medium

A. The _source key is misspelled

B. The includes field should be a list, not a comma-separated string

C. The query is missing a filter clause

D. The match_all query is invalid

Solution

1. Step 1: Check includes value type

   The includes field expects an array of strings, not a single comma-separated string.

2. Step 2: Identify correct format

   It should be ["title", "author"] instead of "title,author".

3. Final Answer:

   The includes field should be a list, not a comma-separated string -> Option B

4. Quick Check:

   includes must be an array, not a string [OK]

Hint: Use arrays for includes/excludes fields, not strings [OK]

Common Mistakes:

• Using comma-separated strings instead of arrays
• Misspelling _source key
• Confusing query errors with source filtering

5. You want to return all fields except password and ssn from documents, but also include only the name and email fields explicitly. Which source filtering configuration achieves this correctly?

hard

A. "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] }

B. "_source": { "include": ["name", "email", "password", "ssn"] }

C. "_source": { "exclude": ["password", "ssn"] }

D. "_source": { "include": ["name", "email"] }

Solution

1. Step 1: Understand combined includes and excludes

   Using both includes and excludes lets you specify fields to return and fields to omit.

2. Step 2: Analyze options

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } includes name and email but excludes password and ssn, matching the requirement.

3. Final Answer:

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } -> Option A

4. Quick Check:

   includes + excludes together filters fields precisely [OK]

Hint: Combine includes and excludes to fine-tune returned fields [OK]

Common Mistakes:

• Using only includes or excludes, missing fields
• Including fields meant to be excluded
• Misunderstanding combined filtering behavior