Field and document level security in Elasticsearch - Step-by-Step Execution

Concept Flow - Field and document level security

User Request

↓

Check Document Level Security

↓

Check Field Level Security

↓

Return Filtered Data

When a user requests data, Elasticsearch first filters documents based on document level security, then filters fields based on field level security, and finally returns the filtered data.

Execution Sample

Elasticsearch

GET /my-index/_search
{
  "query": { "match_all": {} },
  "_source": ["title", "author"],
  "docvalue_fields": ["publish_date"]
}

This query fetches documents from 'my-index' showing only 'title' and 'author' fields, applying field and document level security filters.

Execution Table

Step	Action	Document Filter Applied	Fields Filter Applied	Result
1	Receive user request	No	No	Full documents and fields requested
2	Apply document level security filter	Yes - only docs user can see	No	Documents filtered to allowed subset
3	Apply field level security filter	Yes	Yes - only allowed fields	Fields filtered to allowed subset
4	Return filtered data	Yes	Yes	User receives only allowed documents and fields

💡 All filters applied; data returned respects user security permissions

Variable Tracker

Variable	Start	After Document Filter	After Field Filter	Final
documents	All documents in index	Subset allowed by doc-level security	Same subset	Subset allowed by doc-level security
fields	All fields requested	All fields requested	Subset allowed by field-level security	Subset allowed by field-level security

Key Moments - 2 Insights

Why do some documents not appear even if they match the query?

Because document level security filters out documents the user is not allowed to see, as shown in step 2 of the execution_table.

Why are some fields missing from the returned documents?

Field level security removes fields the user is not permitted to access, as shown in step 3 of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, at which step are documents filtered based on user permissions?

A Step 2

B Step 3

C Step 1

D Step 4

💡 Hint

Check the 'Document Filter Applied' column in the execution_table.

According to variable_tracker, what happens to the 'fields' variable after field level security is applied?

A It remains all requested fields

B It becomes a subset of allowed fields

C It becomes empty

D It includes extra fields

💡 Hint

Look at the 'fields' row in variable_tracker after 'After Field Filter' column.

If document level security is disabled, how would the execution_table change?

A Step 4 would not return any data

B Step 3 would show 'No' for field filter applied

C Step 2 would show 'No' for document filter applied

D Step 1 would apply field filters

💡 Hint

Document level security is applied at step 2 in the execution_table.

Concept Snapshot

Field and document level security in Elasticsearch:
- Document level security filters which documents a user can see.
- Field level security filters which fields in those documents are visible.
- Both filters apply before data is returned.
- Configured in roles and permissions.
- Ensures users see only allowed data.

Full Transcript

When a user sends a request to Elasticsearch, the system first checks document level security to filter out documents the user is not allowed to see. Then, it applies field level security to hide fields the user cannot access within those documents. Finally, Elasticsearch returns only the allowed documents and fields. This process ensures data privacy and security by controlling access at both document and field levels.

Practice

1. What is the main purpose of field-level security in Elasticsearch?

easy

A. To restrict access to specific fields within documents

B. To encrypt the entire Elasticsearch index

C. To limit the number of documents returned in a query

D. To control user login credentials

Solution

1. Step 1: Understand field-level security concept

   Field-level security controls which fields in a document a user can see or query.

2. Step 2: Compare with other options

   Encryption and login control are unrelated to field-level security; limiting documents is document-level security.

3. Final Answer:

   To restrict access to specific fields within documents -> Option A

4. Quick Check:

   Field-level security = restrict fields [OK]

Hint: Field-level security hides fields, not whole documents [OK]

Common Mistakes:

• Confusing field-level with document-level security
• Thinking it encrypts data
• Assuming it controls user passwords

2. Which of the following is the correct syntax to define field-level security in an Elasticsearch role?

easy

A. "fields": ["title", "author"]

B. "field_security": { "deny": ["title", "author"] }

C. "field_security": { "grant": ["title", "author"] }

D. "field_access": { "allow": ["title", "author"] }

Solution

1. Step 1: Recall correct field-level security syntax

   Elasticsearch uses "field_security" with a "grant" array to specify allowed fields.

2. Step 2: Eliminate incorrect options

   "deny" is not valid here; "fields" and "field_access" are incorrect keys.

3. Final Answer:

   "field_security": { "grant": ["title", "author"] } -> Option C

4. Quick Check:

   Use "field_security" with "grant" for allowed fields [OK]

Hint: Use "field_security" with "grant" to allow fields [OK]

Common Mistakes:

• Using "deny" instead of "grant"
• Wrong key names like "fields" or "field_access"
• Confusing syntax with document-level security

3. Given this role definition snippet:

{
  "indices": [
    {
      "names": ["books"],
      "privileges": ["read"],
      "query": { "term": { "genre": "fiction" } },
      "field_security": { "grant": ["title", "author"] }
    }
  ]
}

What documents and fields will a user with this role see when querying the books index?

medium

A. All documents showing all fields

B. All documents showing only 'title' and 'author' fields

C. Only documents where genre is 'fiction' showing all fields

D. Only documents where genre is 'fiction' showing only 'title' and 'author' fields

Solution

1. Step 1: Analyze document-level security query

   The "query" limits documents to those with genre 'fiction'.

2. Step 2: Analyze field-level security grant

   Only "title" and "author" fields are visible due to "field_security".

3. Final Answer:

   Only documents where genre is 'fiction' showing only 'title' and 'author' fields -> Option D

4. Quick Check:

   Query filters docs + grant limits fields = Only documents where genre is 'fiction' showing only 'title' and 'author' fields [OK]

Hint: Query filters docs; field_security limits fields shown [OK]

Common Mistakes:

• Ignoring the query filter on documents
• Assuming all fields are visible
• Confusing document and field level restrictions

4. You defined this role snippet:

{
  "indices": [
    {
      "names": ["library"],
      "privileges": ["read"],
      "query": { "term": { "category": "science" } },
      "field_security": { "grant": ["title", "summary"] }
    }
  ]
}

But users report they see all documents and fields. What is the likely error?

medium

A. The query filter is incorrect or not applied properly

B. Field names in grant are misspelled

C. Privileges should include "write" to restrict fields

D. Role must include "manage" privilege for security to work

Solution

1. Step 1: Check query filter correctness

   If the query filter is malformed or ignored, document filtering won't happen.

2. Step 2: Verify field_security and privileges

   Field names look correct; "read" privilege is enough for filtering; "write" or "manage" not needed.

3. Final Answer:

   The query filter is incorrect or not applied properly -> Option A

4. Quick Check:

   Query filter controls docs; if ignored, all docs show [OK]

Hint: Check query syntax if document filtering fails [OK]

Common Mistakes:

• Assuming 'write' privilege needed for filtering
• Ignoring query filter syntax errors
• Thinking field names cause document filtering issues

5. You want to create a role that allows users to read only documents where status is active and see only the name and email fields. Which role definition snippet correctly implements this?

hard

A. {
"indices": [
{
"names": ["users"],
"privileges": ["read"],
"query": { "match": { "status": "active" } },
"field_security": { "deny": ["password"] }
}
]
}

B. {
"indices": [
{
"names": ["users"],
"privileges": ["read"],
"query": { "term": { "status": "active" } },
"field_security": { "grant": ["name", "email"] }
}
]
}

C. {
"indices": [
{
"names": ["users"],
"privileges": ["read"],
"query": { "term": { "status": "active" } },
"fields": ["name", "email"]
}
]
}

D. {
"indices": [
{
"names": ["users"],
"privileges": ["read"],
"query": { "term": { "status": "active" } }
}
]
}

Solution

1. Step 1: Verify document-level security query

   Using "term" query on "status" with "active" correctly filters documents.

2. Step 2: Verify field-level security syntax

   "field_security" with "grant" array specifying "name" and "email" fields is correct.

3. Step 3: Eliminate incorrect options

   { "indices": [ { "names": ["users"], "privileges": ["read"], "query": { "match": { "status": "active" } }, "field_security": { "deny": ["password"] } } ] } uses "deny" which is invalid; { "indices": [ { "names": ["users"], "privileges": ["read"], "query": { "term": { "status": "active" } }, "fields": ["name", "email"] } ] } uses wrong key "fields"; { "indices": [ { "names": ["users"], "privileges": ["read"], "query": { "term": { "status": "active" } } } ] } lacks field-level security.

4. Final Answer:

   Role with "query" term filter and "field_security" grant for "name" and "email" -> Option B

5. Quick Check:

   Use "query" for docs + "field_security" grant for fields [OK]

Hint: Use "query" for docs and "field_security" with "grant" for fields [OK]

Common Mistakes:

• Using "deny" instead of "grant" in field_security
• Using wrong keys like "fields" instead of "field_security"
• Omitting field-level security to restrict fields