Runtime fields in Elasticsearch - Step-by-Step Execution

Concept Flow - Runtime fields

Search Request Received

↓

Check for Runtime Fields

↓

Evaluate Script

↓

Add Runtime Fields to Result

↓

Return Results

When a search request includes runtime fields, Elasticsearch evaluates their scripts on the fly and adds the computed values to the search results.

Execution Sample

Elasticsearch

{
  "runtime_mappings": {
    "day_of_week": {
      "type": "keyword",
      "script": "emit(doc['date'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ENGLISH))"
    }
  }
}

Defines a runtime field 'day_of_week' that computes the day name from a date field during search.

Execution Table

Step	Action	Input	Script Evaluation	Result
1	Receive search request	{"runtime_mappings": {"day_of_week": {...}}}	N/A	Start processing
2	Check for runtime fields	runtime_mappings present	N/A	Proceed to evaluate scripts
3	For each document	doc['date'] value	Evaluate dayOfWeekEnum.getDisplayName()	Emit day name string
4	Add runtime field to document	Emitted day name	N/A	Document enriched with 'day_of_week'
5	Return search results	All documents with runtime fields	N/A	Results sent to client

💡 All documents processed; runtime fields evaluated and added to results

Variable Tracker

Variable	Start	After Doc 1	After Doc 2	Final
doc['date'].value	N/A	2024-06-01	2024-06-02	Last document date
day_of_week (emitted)	N/A	Saturday	Sunday	Last emitted day

Key Moments - 2 Insights

Why does the runtime field script run for each document?

Because runtime fields are computed on the fly during search, the script runs for every document to produce the field's value, as shown in execution_table step 3.

Does the runtime field change the stored data in Elasticsearch?

No, runtime fields do not modify stored data; they only add computed values at query time, as seen in step 4 where the field is added to the result but not stored.

Visual Quiz - 3 Questions

Look at the execution_table, at which step does Elasticsearch evaluate the runtime field script?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Check the 'Script Evaluation' column in execution_table rows

According to variable_tracker, what is the emitted value for the first document's runtime field?

A Saturday

B Sunday

C Monday

D Friday

💡 Hint

Look at 'day_of_week (emitted)' after Doc 1 in variable_tracker

If the runtime field script was removed, what would happen to the search results?

A Search would fail with an error

B Results would include the runtime field with default values

C Results would not include the runtime field

D Stored fields would be overwritten

💡 Hint

Refer to concept_flow where runtime fields are checked and evaluated only if present

Concept Snapshot

Runtime fields let you add fields computed at search time.
Define them with scripts in runtime_mappings.
Scripts run for each document during search.
They do not change stored data.
Results include these computed fields dynamically.

Full Transcript

Runtime fields in Elasticsearch are fields computed during search using scripts. When a search request includes runtime fields, Elasticsearch checks for them and evaluates their scripts for each document. For example, a runtime field can compute the day of the week from a date field. The script runs for every document, emitting the computed value, which is then added to the search result. This process does not modify stored data but enriches the results dynamically. The execution table shows the step-by-step flow from receiving the request, checking for runtime fields, evaluating scripts per document, adding the computed fields, and returning results. The variable tracker shows how document dates and emitted day names change during execution. Key moments clarify why scripts run per document and that stored data remains unchanged. The visual quiz tests understanding of when scripts run, emitted values, and effects of removing runtime fields.

Practice

1. What is the main purpose of runtime fields in Elasticsearch?

easy

A. To backup the index data automatically

B. To create new fields dynamically during search without changing stored data

C. To delete existing fields from documents

D. To permanently add new fields to the index mapping

Solution

1. Step 1: Understand runtime fields concept

   Runtime fields are used to add fields dynamically at query time without modifying the stored data.

2. Step 2: Compare options with concept

   Only To create new fields dynamically during search without changing stored data describes creating fields dynamically during search without changing stored data.

3. Final Answer:

   To create new fields dynamically during search without changing stored data -> Option B

4. Quick Check:

   Runtime fields = dynamic fields at search time [OK]

Hint: Runtime fields add data on-the-fly, not stored permanently [OK]

Common Mistakes:

• Confusing runtime fields with permanent mapping changes
• Thinking runtime fields modify stored documents
• Assuming runtime fields delete data

2. Which of the following is the correct syntax to define a runtime field named full_name that concatenates first_name and last_name using painless script?

easy

A. {
"runtime_mappings": {
"full_name": {
"type": "keyword",
"script": "return doc['first_name'] + ' ' + doc['last_name']"
}
}
}

B. {
"mappings": {
"full_name": {
"type": "text"
}
}
}

C. {
"runtime_fields": {
"full_name": {
"type": "keyword",
"script": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)"
}
}
}

D. {
"runtime_mappings": {
"full_name": {
"type": "keyword",
"script": {
"source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)"
}
}
}
}

Solution

1. Step 1: Identify correct runtime field syntax

   Runtime fields are defined under runtime_mappings with a type and a script object containing source code.

2. Step 2: Check script correctness

   { "runtime_mappings": { "full_name": { "type": "keyword", "script": { "source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)" } } } } uses emit() inside source string and accesses doc['field'].value correctly.

3. Final Answer:

   { "runtime_mappings": { "full_name": { "type": "keyword", "script": { "source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)" } } } } -> Option D

4. Quick Check:

   runtime_mappings + emit() + doc['field'].value = correct syntax [OK]

Hint: Use runtime_mappings with script source and emit() for runtime fields [OK]

Common Mistakes:

• Using mappings instead of runtime_mappings
• Missing emit() function in script
• Incorrect script syntax without source object

3. Given this runtime field definition in a search query:

{
  "runtime_mappings": {
    "age_plus_ten": {
      "type": "long",
      "script": {
        "source": "emit(doc['age'].value + 10)"
      }
    }
  }
}

What will be the value of age_plus_ten for a document with age = 25?

medium

A. 35

B. 15

C. 25

D. Error: field not found

Solution

1. Step 1: Understand the script logic

   The script emits the value of age field plus 10.

2. Step 2: Calculate the result for age=25

   25 + 10 = 35.

3. Final Answer:

   35 -> Option A

4. Quick Check:

   age + 10 = 35 [OK]

Hint: Add 10 to age field value as scripted [OK]

Common Mistakes:

• Confusing addition with subtraction
• Assuming runtime fields modify stored data
• Expecting syntax error instead of calculation

4. You wrote this runtime field script:

{
  "runtime_mappings": {
    "discounted_price": {
      "type": "double",
      "script": {
        "source": "emit(doc['price'].value * 0.9)"
      }
    }
  }
}

But the query fails with an error: Field [price] not found in doc. What is the likely cause?

medium

A. Runtime fields cannot use numeric types

B. The script syntax is incorrect

C. The price field is missing in some documents

D. The discounted_price field must be defined in mappings

Solution

1. Step 1: Analyze error message

   Error says price field not found in document, meaning some docs lack this field.

2. Step 2: Understand runtime field behavior

   Runtime scripts fail if they access missing fields without checks.

3. Final Answer:

   The price field is missing in some documents -> Option C

4. Quick Check:

   Missing field in doc causes runtime script error [OK]

Hint: Check if all docs have fields used in runtime scripts [OK]

Common Mistakes:

• Assuming script syntax error without checking data
• Thinking runtime fields require mapping changes
• Ignoring missing field presence in documents

5. You want to create a runtime field status that returns "adult" if age ≥ 18, otherwise "minor". Which painless script correctly implements this logic?

hard

A. "emit(doc['age'].value >= 18 ? 'adult' : 'minor')"

B. "if (doc['age'].value >= 18) { return 'adult' } else { return 'minor' }"

C. "emit(doc['age'] >= 18 ? 'adult' : 'minor')"

D. "emit(doc['age'].value > 18 ? 'adult' : 'minor')"

Solution

1. Step 1: Check correct painless syntax for runtime fields

   Runtime fields use emit() to output values; accessing field value requires doc['age'].value.

2. Step 2: Verify conditional logic

   "emit(doc['age'].value >= 18 ? 'adult' : 'minor')" uses ternary operator with >= 18 and emits 'adult' or 'minor' correctly.

3. Final Answer:

   "emit(doc['age'].value >= 18 ? 'adult' : 'minor')" -> Option A

4. Quick Check:

   emit() + ternary + doc['age'].value = correct [OK]

Hint: Use emit() with ternary and doc['field'].value for conditions [OK]

Common Mistakes:

• Using return instead of emit() in runtime fields
• Accessing doc['age'] without .value
• Using > instead of >= changing logic