Source filtering in Elasticsearch

Introduction

Source filtering helps you get only the parts of data you want from a search result. This saves time and makes your results easier to use.

You want to get only a few fields from a large document to reduce data size.

You want to hide sensitive information like passwords or personal details from search results.

You want to speed up your app by fetching only the needed data.

You want to simplify the data before sending it to another system or user.

You want to avoid transferring large documents over the network.

Syntax

Elasticsearch

{
  "_source": ["field1", "field2", ...],
  "query": {
    ... your query ...
  }
}

You can list fields to include only those in the results.

You can also exclude fields by using an object with includes and excludes arrays.

Examples

This example returns only the title and author fields from each document.

Elasticsearch

{
  "_source": ["title", "author"]
}

This example includes title and author but excludes the comments field.

Elasticsearch

{
  "_source": {
    "includes": ["title", "author"],
    "excludes": ["comments"]
  }
}

This example disables source fetching completely, so no document fields are returned.

Elasticsearch

{
  "_source": false,
  "query": {
    "match_all": {}
  }
}

Sample Program

This search finds documents where the city is London and returns only the name and age fields.

Elasticsearch

{
  "_source": ["name", "age"],
  "query": {
    "match": {
      "city": "London"
    }
  }
}

Important Notes

Source filtering only affects what fields are returned, not what documents are found.

If you exclude all fields, you get no document data, just metadata like IDs.

Use source filtering to reduce network load and improve performance.

Summary

Source filtering controls which parts of documents you get back from a search.

You can include or exclude specific fields to get just what you need.

This helps make your app faster and your data easier to handle.

Practice

1. What is the main purpose of _source filtering in Elasticsearch?

easy

A. To control which parts of the document are returned in search results

B. To filter documents before indexing

C. To change the document content during indexing

D. To sort search results by field values

Solution

1. Step 1: Understand _source filtering role

   _source filtering controls which fields of a document are included in the search response.

2. Step 2: Differentiate from other features

   It does not affect indexing or sorting, only what data is returned after a search.

3. Final Answer:

   To control which parts of the document are returned in search results -> Option A

4. Quick Check:

   Source filtering = control returned fields [OK]

Hint: Remember: source filtering limits returned fields only [OK]

Common Mistakes:

• Confusing source filtering with indexing filters
• Thinking it changes document content
• Assuming it sorts results

2. Which of the following is the correct syntax to exclude the field password from the search results using source filtering?

easy

A. "_source": { "filter": ["password"] }

B. "_source": { "include": ["password"] }

C. "_source": { "excludes": ["password"] }

D. "_source": { "remove": ["password"] }

Solution

1. Step 1: Recall source filtering syntax

   To exclude fields, use "excludes" inside _source with a list of field names.

2. Step 2: Match correct syntax

   "_source": { "excludes": ["password"] } uses "excludes": ["password"], which is the correct way to exclude the password field.

3. Final Answer:

   "_source": { "excludes": ["password"] } -> Option C

4. Quick Check:

   Exclude fields = use "excludes" key [OK]

Hint: Use "excludes" key to remove fields from results [OK]

Common Mistakes:

• Using "include" to exclude fields
• Using non-existent keys like "filter" or "remove"
• Wrong JSON structure

3. Given this Elasticsearch query snippet, what fields will be returned in the search results?

{
  "_source": { "includes": ["title", "author"] },
  "query": { "match_all": {} }
}

medium

A. All fields will be returned

B. All fields except title and author will be returned

C. No fields will be returned

D. Only the fields title and author will be returned

Solution

1. Step 1: Understand includes in _source

   The includes key specifies which fields to return from the document.

2. Step 2: Apply to given query

   Since only title and author are included, only these fields will appear in the results.

3. Final Answer:

   Only the fields title and author will be returned -> Option D

4. Quick Check:

   includes fields = return only those fields [OK]

Hint: includes means return only listed fields [OK]

Common Mistakes:

• Thinking includes excludes fields
• Assuming all fields return by default
• Confusing includes with excludes

4. Identify the error in this source filtering snippet:

{
  "_source": {
    "includes": "title,author"
  },
  "query": { "match_all": {} }
}

medium

A. The _source key is misspelled

B. The includes field should be a list, not a comma-separated string

C. The query is missing a filter clause

D. The match_all query is invalid

Solution

1. Step 1: Check includes value type

   The includes field expects an array of strings, not a single comma-separated string.

2. Step 2: Identify correct format

   It should be ["title", "author"] instead of "title,author".

3. Final Answer:

   The includes field should be a list, not a comma-separated string -> Option B

4. Quick Check:

   includes must be an array, not a string [OK]

Hint: Use arrays for includes/excludes fields, not strings [OK]

Common Mistakes:

• Using comma-separated strings instead of arrays
• Misspelling _source key
• Confusing query errors with source filtering

5. You want to return all fields except password and ssn from documents, but also include only the name and email fields explicitly. Which source filtering configuration achieves this correctly?

hard

A. "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] }

B. "_source": { "include": ["name", "email", "password", "ssn"] }

C. "_source": { "exclude": ["password", "ssn"] }

D. "_source": { "include": ["name", "email"] }

Solution

1. Step 1: Understand combined includes and excludes

   Using both includes and excludes lets you specify fields to return and fields to omit.

2. Step 2: Analyze options

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } includes name and email but excludes password and ssn, matching the requirement.

3. Final Answer:

   "_source": { "includes": ["name", "email"], "excludes": ["password", "ssn"] } -> Option A

4. Quick Check:

   includes + excludes together filters fields precisely [OK]

Hint: Combine includes and excludes to fine-tune returned fields [OK]

Common Mistakes:

• Using only includes or excludes, missing fields
• Including fields meant to be excluded
• Misunderstanding combined filtering behavior