Source filtering in Elasticsearch - Step-by-Step Execution

Concept Flow - Source filtering

Send Query with _source filter

↓

Elasticsearch receives query

↓

Check _source filter fields

↓

Include fields

↓

Fetch only specified fields

↓

Return filtered results

Source filtering tells Elasticsearch which fields to include or exclude in the search results, reducing data size.

Execution Sample

Elasticsearch

{
  "query": { "match_all": {} },
  "_source": ["title", "author"]
}

This query fetches all documents but returns only the 'title' and 'author' fields.

Execution Table

Step	Action	Input	Result
1	Receive query	{"query":{"match_all":{}},"_source":["title","author"]}	Query parsed with _source filter
2	Check _source filter	["title", "author"]	Fields to include: title, author
3	Fetch documents	All documents matching match_all	Documents fetched with all fields
4	Apply source filtering	Documents with all fields	Documents with only title and author fields
5	Return results	Filtered documents	Response contains only title and author fields
6	End	No more steps	Execution complete

💡 All matching documents returned with only specified fields included

Variable Tracker

Variable	Start	After Step 2	After Step 4	Final
_source filter	undefined	["title", "author"]	["title", "author"]	["title", "author"]
Documents fetched	none	none	only title and author fields	only title and author fields

Key Moments - 3 Insights

Why do only 'title' and 'author' fields appear in the results?

Because the _source filter includes only these fields as shown in execution_table step 4, Elasticsearch excludes all other fields.

What happens if _source filter is omitted?

If omitted, Elasticsearch returns all fields of the documents, as no filtering is applied (not shown in this trace).

Can _source filter exclude fields instead of including?

Yes, you can specify fields to exclude using _source with 'excludes' option, but this example uses inclusion only.

Visual Quiz - 3 Questions

Look at the execution_table, what fields are included after step 4?

A ["title", "author"]

B ["title", "author", "date"]

C All fields

D No fields

💡 Hint

Check the 'Result' column at step 4 in execution_table

At which step does Elasticsearch apply the source filtering?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Look for the step where documents are filtered to only specified fields in execution_table

If the _source filter was removed, how would the 'Documents fetched' variable change after step 4?

A It would remain only title and author fields

B It would be empty

C It would include all fields

D It would include only excluded fields

💡 Hint

Refer to key_moments about what happens if _source filter is omitted

Concept Snapshot

Source filtering in Elasticsearch lets you specify which fields to include or exclude in search results.
Use the _source parameter with an array of fields to include or an object with includes/excludes.
This reduces data size and improves performance by returning only needed fields.
Example: {"_source": ["title", "author"]} returns only those fields.
If omitted, all fields are returned by default.

Full Transcript

Source filtering is a way to tell Elasticsearch which fields you want in your search results. When you send a query with the _source filter, Elasticsearch reads the fields you want to include or exclude. It fetches all matching documents but only returns the specified fields. This makes the response smaller and faster to handle. For example, if you ask for only 'title' and 'author', Elasticsearch will return documents with just those fields. If you don't use source filtering, all fields come back by default. This trace shows each step from receiving the query, checking the filter, fetching documents, applying the filter, and returning the results.

Practice

1. What is the main purpose of _source filtering in Elasticsearch?

easy

A. To control which parts of the document are returned in search results

B. To filter documents before indexing

C. To change the document content during indexing

D. To sort search results by field values

Solution

1. Step 1: Understand _source filtering role

   _source filtering controls which fields of a document are included in the search response.

2. Step 2: Differentiate from other features

   It does not affect indexing or sorting, only what data is returned after a search.

3. Final Answer:

   To control which parts of the document are returned in search results -> Option A

4. Quick Check:

   Source filtering = control returned fields [OK]

Hint: Remember: source filtering limits returned fields only [OK]

Common Mistakes:

• Confusing source filtering with indexing filters
• Thinking it changes document content
• Assuming it sorts results

2. Which of the following is the correct syntax to exclude the field password from the search results using source filtering?

easy

A. "_source": { "filter": ["password"] }

B. "_source": { "include": ["password"] }

C. "_source": { "excludes": ["password"] }

D. "_source": { "remove": ["password"] }

Solution

1. Step 1: Recall source filtering syntax

   To exclude fields, use "excludes" inside _source with a list of field names.

2. Step 2: Match correct syntax

   "_source": { "excludes": ["password"] } uses "excludes": ["password"], which is the correct way to exclude the password field.

3. Final Answer:

   "_source": { "excludes": ["password"] } -> Option C

4. Quick Check:

   Exclude fields = use "excludes" key [OK]

Hint: Use "excludes" key to remove fields from results [OK]

Common Mistakes:

• Using "include" to exclude fields
• Using non-existent keys like "filter" or "remove"
• Wrong JSON structure

3. Given this Elasticsearch query snippet, what fields will be returned in the search results?

{
  "_source": { "includes": ["title", "author"] },
  "query": { "match_all": {} }
}

medium

A. All fields will be returned

B. All fields except title and author will be returned

C. No fields will be returned

D. Only the fields title and author will be returned

Solution

1. Step 1: Understand includes in _source

   The includes key specifies which fields to return from the document.

2. Step 2: Apply to given query

   Since only title and author are included, only these fields will appear in the results.

3. Final Answer:

   Only the fields title and author will be returned -> Option D

4. Quick Check:

   includes fields = return only those fields [OK]

Hint: includes means return only listed fields [OK]

Common Mistakes:

• Thinking includes excludes fields
• Assuming all fields return by default
• Confusing includes with excludes

4. Identify the error in this source filtering snippet:

{
  "_source": {
    "includes": "title,author"
  },
  "query": { "match_all": {} }
}

medium

A. The _source key is misspelled

B. The includes field should be a list, not a comma-separated string

C. The query is missing a filter clause

D. The match_all query is invalid

Solution

1. Step 1: Check includes value type

   The includes field expects an array of strings, not a single comma-separated string.

2. Step 2: Identify correct format

   It should be ["title", "author"] instead of "title,author".

3. Final Answer:

   The includes field should be a list, not a comma-separated string -> Option B

4. Quick Check:

   includes must be an array, not a string [OK]

Hint: Use arrays for includes/excludes fields, not strings [OK]

Common Mistakes:

• Using comma-separated strings instead of arrays
• Misspelling _source key
• Confusing query errors with source filtering

5. You want to return all fields except password and ssn from documents, but also include only the name and email fields explicitly. Which source filtering configuration achieves this correctly?

hard

A. "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] }

B. "_source": { "include": ["name", "email", "password", "ssn"] }

C. "_source": { "exclude": ["password", "ssn"] }

D. "_source": { "include": ["name", "email"] }

Solution

1. Step 1: Understand combined includes and excludes

   Using both includes and excludes lets you specify fields to return and fields to omit.

2. Step 2: Analyze options

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } includes name and email but excludes password and ssn, matching the requirement.

3. Final Answer:

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } -> Option A

4. Quick Check:

   includes + excludes together filters fields precisely [OK]

Hint: Combine includes and excludes to fine-tune returned fields [OK]

Common Mistakes:

• Using only includes or excludes, missing fields
• Including fields meant to be excluded
• Misunderstanding combined filtering behavior