Runtime fields in Elasticsearch - Mini Project: Build & Apply

Using Runtime Fields in Elasticsearch

📖 Scenario: You have a collection of products indexed in Elasticsearch. Each product has a price and quantity field. You want to calculate the total_value of each product (price multiplied by quantity) on the fly without changing the original data.

🎯 Goal: Create a runtime field called total_value that calculates the product of price and quantity for each document. Then, query the index to show the name and total_value for each product.

📋 What You'll Learn

Create an index mapping with fields name (keyword), price (double), and quantity (integer).

Add a runtime field total_value that multiplies price and quantity.

Index at least 3 product documents with exact values.

Query the index to return name and total_value fields.

💡 Why This Matters

🌍 Real World

Runtime fields help you add new calculated data to your Elasticsearch queries without reindexing your data. This is useful when you want quick insights or temporary calculations.

💼 Career

Many jobs working with Elasticsearch require knowledge of runtime fields to optimize queries and add dynamic data without downtime or heavy reindexing.

1

Create the index with product data

Create an index called products with the following documents exactly: {"name": "Pen", "price": 1.5, "quantity": 10}, {"name": "Notebook", "price": 3.0, "quantity": 5}, and {"name": "Eraser", "price": 0.5, "quantity": 20}. Use the _bulk API to add these documents.

Elasticsearch

# Use the _bulk API to index the 3 products
# Your code here

Hint

Use the _bulk API with { "index": {} } lines before each document.

2

Add a runtime field for total value

Update the products index mapping to add a runtime field called total_value of type double. This field should calculate doc['price'].value * doc['quantity'].value.

Elasticsearch

POST /products/_bulk
{ "index": {} }
{ "name": "Pen", "price": 1.5, "quantity": 10 }
{ "index": {} }
{ "name": "Notebook", "price": 3.0, "quantity": 5 }
{ "index": {} }
{ "name": "Eraser", "price": 0.5, "quantity": 20 }

# Add runtime field mapping below
# Your code here

Hint

Use the PUT /products/_mapping API with a runtime field named total_value and a script that multiplies price and quantity.

3

Query products with runtime field

Write a search query on the products index that returns the name and the runtime field total_value for each product. Use _source to include name and runtime_mappings to include total_value.

Elasticsearch

POST /products/_bulk
{ "index": {} }
{ "name": "Pen", "price": 1.5, "quantity": 10 }
{ "index": {} }
{ "name": "Notebook", "price": 3.0, "quantity": 5 }
{ "index": {} }
{ "name": "Eraser", "price": 0.5, "quantity": 20 }

PUT /products/_mapping
{
  "runtime": {
    "total_value": {
      "type": "double",
      "script": {
        "source": "emit(doc['price'].value * doc['quantity'].value)"
      }
    }
  }
}

# Write the search query below
# Your code here

Hint

Use _source to get name, add runtime_mappings for total_value, and request fields to show total_value.

4

Display the product names and total values

Print the name and total_value for each product from the search results. Format each line as: Product: <name>, Total Value: <total_value>.

Elasticsearch

POST /products/_bulk
{ "index": {} }
{ "name": "Pen", "price": 1.5, "quantity": 10 }
{ "index": {} }
{ "name": "Notebook", "price": 3.0, "quantity": 5 }
{ "index": {} }
{ "name": "Eraser", "price": 0.5, "quantity": 20 }

PUT /products/_mapping
{
  "runtime": {
    "total_value": {
      "type": "double",
      "script": {
        "source": "emit(doc['price'].value * doc['quantity'].value)"
      }
    }
  }
}

GET /products/_search
{
  "_source": ["name"],
  "runtime_mappings": {
    "total_value": {
      "type": "double",
      "script": {
        "source": "emit(doc['price'].value * doc['quantity'].value)"
      }
    }
  },
  "fields": ["total_value"]
}

# Print each product's name and total_value in the format:
# Product: , Total Value: 
# Your code here

Hint

Print each product's name and total_value exactly as shown.

Practice

1. What is the main purpose of runtime fields in Elasticsearch?

easy

A. To backup the index data automatically

B. To create new fields dynamically during search without changing stored data

C. To delete existing fields from documents

D. To permanently add new fields to the index mapping

Solution

1. Step 1: Understand runtime fields concept

   Runtime fields are used to add fields dynamically at query time without modifying the stored data.

2. Step 2: Compare options with concept

   Only To create new fields dynamically during search without changing stored data describes creating fields dynamically during search without changing stored data.

3. Final Answer:

   To create new fields dynamically during search without changing stored data -> Option B

4. Quick Check:

   Runtime fields = dynamic fields at search time [OK]

Hint: Runtime fields add data on-the-fly, not stored permanently [OK]

Common Mistakes:

• Confusing runtime fields with permanent mapping changes
• Thinking runtime fields modify stored documents
• Assuming runtime fields delete data

2. Which of the following is the correct syntax to define a runtime field named full_name that concatenates first_name and last_name using painless script?

easy

A. {
"runtime_mappings": {
"full_name": {
"type": "keyword",
"script": "return doc['first_name'] + ' ' + doc['last_name']"
}
}
}

B. {
"mappings": {
"full_name": {
"type": "text"
}
}
}

C. {
"runtime_fields": {
"full_name": {
"type": "keyword",
"script": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)"
}
}
}

D. {
"runtime_mappings": {
"full_name": {
"type": "keyword",
"script": {
"source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)"
}
}
}
}

Solution

1. Step 1: Identify correct runtime field syntax

   Runtime fields are defined under runtime_mappings with a type and a script object containing source code.

2. Step 2: Check script correctness

   { "runtime_mappings": { "full_name": { "type": "keyword", "script": { "source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)" } } } } uses emit() inside source string and accesses doc['field'].value correctly.

3. Final Answer:

   { "runtime_mappings": { "full_name": { "type": "keyword", "script": { "source": "emit(doc['first_name'].value + ' ' + doc['last_name'].value)" } } } } -> Option D

4. Quick Check:

   runtime_mappings + emit() + doc['field'].value = correct syntax [OK]

Hint: Use runtime_mappings with script source and emit() for runtime fields [OK]

Common Mistakes:

• Using mappings instead of runtime_mappings
• Missing emit() function in script
• Incorrect script syntax without source object

3. Given this runtime field definition in a search query:

{
  "runtime_mappings": {
    "age_plus_ten": {
      "type": "long",
      "script": {
        "source": "emit(doc['age'].value + 10)"
      }
    }
  }
}

What will be the value of age_plus_ten for a document with age = 25?

medium

A. 35

B. 15

C. 25

D. Error: field not found

Solution

1. Step 1: Understand the script logic

   The script emits the value of age field plus 10.

2. Step 2: Calculate the result for age=25

   25 + 10 = 35.

3. Final Answer:

   35 -> Option A

4. Quick Check:

   age + 10 = 35 [OK]

Hint: Add 10 to age field value as scripted [OK]

Common Mistakes:

• Confusing addition with subtraction
• Assuming runtime fields modify stored data
• Expecting syntax error instead of calculation

4. You wrote this runtime field script:

{
  "runtime_mappings": {
    "discounted_price": {
      "type": "double",
      "script": {
        "source": "emit(doc['price'].value * 0.9)"
      }
    }
  }
}

But the query fails with an error: Field [price] not found in doc. What is the likely cause?

medium

A. Runtime fields cannot use numeric types

B. The script syntax is incorrect

C. The price field is missing in some documents

D. The discounted_price field must be defined in mappings

Solution

1. Step 1: Analyze error message

   Error says price field not found in document, meaning some docs lack this field.

2. Step 2: Understand runtime field behavior

   Runtime scripts fail if they access missing fields without checks.

3. Final Answer:

   The price field is missing in some documents -> Option C

4. Quick Check:

   Missing field in doc causes runtime script error [OK]

Hint: Check if all docs have fields used in runtime scripts [OK]

Common Mistakes:

• Assuming script syntax error without checking data
• Thinking runtime fields require mapping changes
• Ignoring missing field presence in documents

5. You want to create a runtime field status that returns "adult" if age ≥ 18, otherwise "minor". Which painless script correctly implements this logic?

hard

A. "emit(doc['age'].value >= 18 ? 'adult' : 'minor')"

B. "if (doc['age'].value >= 18) { return 'adult' } else { return 'minor' }"

C. "emit(doc['age'] >= 18 ? 'adult' : 'minor')"

D. "emit(doc['age'].value > 18 ? 'adult' : 'minor')"

Solution

1. Step 1: Check correct painless syntax for runtime fields

   Runtime fields use emit() to output values; accessing field value requires doc['age'].value.

2. Step 2: Verify conditional logic

   "emit(doc['age'].value >= 18 ? 'adult' : 'minor')" uses ternary operator with >= 18 and emits 'adult' or 'minor' correctly.

3. Final Answer:

   "emit(doc['age'].value >= 18 ? 'adult' : 'minor')" -> Option A

4. Quick Check:

   emit() + ternary + doc['age'].value = correct [OK]

Hint: Use emit() with ternary and doc['field'].value for conditions [OK]

Common Mistakes:

• Using return instead of emit() in runtime fields
• Accessing doc['age'] without .value
• Using > instead of >= changing logic