Dynamic templates in Elasticsearch

Introduction

Dynamic templates help you tell Elasticsearch how to automatically set the type and settings for fields when new data comes in. This saves time and keeps your data organized without manually defining every field.

When you want Elasticsearch to automatically detect and map new fields in your data.

When your data has many fields with similar patterns and you want to apply the same settings to them.

When you want to control how unknown or new fields are indexed without writing full mappings.

When you want to avoid errors from unexpected field types in your documents.

When you want to optimize search and storage by setting specific types for dynamic fields.

Syntax

Elasticsearch

{
  "mappings": {
    "dynamic_templates": [
      {
        "template_name": {
          "match": "pattern*",
          "mapping": {
            "type": "type_name",
            "other_setting": "value"
          }
        }
      }
    ]
  }
}

dynamic_templates is an array of templates.

Each template has a name and defines a match pattern and the mapping to apply.

Examples

This template maps all string fields to the keyword type automatically.

Elasticsearch

{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keywords": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  }
}

This template matches any field ending with _date and maps it as a date with a specific format.

Elasticsearch

{
  "mappings": {
    "dynamic_templates": [
      {
        "dates": {
          "match": "*_date",
          "mapping": {
            "type": "date",
            "format": "yyyy-MM-dd"
          }
        }
      }
    ]
  }
}

This template maps fields starting with id_ as long integers.

Elasticsearch

{
  "mappings": {
    "dynamic_templates": [
      {
        "long_fields": {
          "match": "id_*",
          "mapping": {
            "type": "long"
          }
        }
      }
    ]
  }
}

Sample Program

This example shows two dynamic templates: one that maps all string fields as keyword type, and another that maps fields ending with _date as dates with a specific format. When you index a document with these fields, Elasticsearch applies these rules automatically.

Elasticsearch

{
  "mappings": {
    "dynamic_templates": [
      {
        "strings_as_keywords": {
          "match_mapping_type": "string",
          "mapping": {
            "type": "keyword"
          }
        }
      },
      {
        "dates": {
          "match": "*_date",
          "mapping": {
            "type": "date",
            "format": "yyyy-MM-dd"
          }
        }
      }
    ]
  }
}

// Example document to index:
// {
//   "user": "alice",
//   "signup_date": "2024-06-01",
//   "comment": "hello world"
// }

// Resulting mapping:
// "user" and "comment" fields are mapped as keyword (because they are strings)
// "signup_date" is mapped as date with format yyyy-MM-dd

Important Notes

Dynamic templates are checked in order, so the first matching template is applied.

You can use match, unmatch, match_mapping_type to control which fields the template applies to.

Dynamic templates help avoid mapping conflicts and make your index flexible.

Summary

Dynamic templates let Elasticsearch automatically map new fields based on rules you set.

They save time and keep your data organized without manual mapping for every field.

Use patterns to match field names or types and define how they should be indexed.

Practice

1. What is the main purpose of dynamic_templates in Elasticsearch?

easy

A. To automatically map new fields based on user-defined rules

B. To manually define each field's mapping one by one

C. To delete old indexes automatically

D. To backup data to another cluster

Solution

1. Step 1: Understand dynamic templates concept

   Dynamic templates let Elasticsearch automatically apply mapping rules to new fields without manual intervention.

2. Step 2: Compare options with concept

   Only To automatically map new fields based on user-defined rules describes automatic mapping based on rules, which matches the purpose of dynamic templates.

3. Final Answer:

   To automatically map new fields based on user-defined rules -> Option A

4. Quick Check:

   Dynamic templates = automatic mapping rules [OK]

Hint: Dynamic templates automate field mapping using rules [OK]

Common Mistakes:

• Thinking dynamic templates delete or backup data
• Confusing manual mapping with dynamic templates
• Assuming dynamic templates only work for existing fields

2. Which of the following is the correct syntax to define a dynamic template that matches fields ending with _date and maps them as date type?

easy

A. "dynamic_templates": [{"dates": {"match": "*_date", "mapping": {"type": "date"}}}]

B. "dynamic_templates": [{"dates": {"match_pattern": "regex", "match": ".*_date$", "mapping": {"type": "date"}}}]

C. "dynamic_templates": [{"dates": {"match": "date", "mapping": {"type": "date"}}}]

D. "dynamic_templates": [{"dates": {"path_match": "*_date", "mapping": {"type": "date"}}}]

Solution

1. Step 1: Identify correct syntax for regex matching

   To match fields ending with '_date', we use match_pattern": "regex" and match": ".*_date$".

2. Step 2: Verify mapping type

   The mapping type should be date as required.

3. Final Answer:

   "dynamic_templates": [{"dates": {"match_pattern": "regex", "match": ".*_date$", "mapping": {"type": "date"}}}] -> Option B

4. Quick Check:

   Regex match + date type = "dynamic_templates": [{"dates": {"match_pattern": "regex", "match": ".*_date$", "mapping": {"type": "date"}}}] [OK]

Hint: Use match_pattern: regex for pattern matching [OK]

Common Mistakes:

• Using match without match_pattern for regex
• Using path_match instead of match for field names
• Incorrect regex syntax or missing $ anchor

3. Given this dynamic template snippet in Elasticsearch mapping:

{
  "dynamic_templates": [
    {
      "strings_as_keywords": {
        "match_mapping_type": "string",
        "mapping": {
          "type": "keyword"
        }
      }
    }
  ]
}

What will be the type of a newly indexed field named username with string value?

medium

A. "text"

B. "object"

C. "keyword"

D. "string"

Solution

1. Step 1: Analyze the dynamic template rule

   The template matches fields of type "string" and maps them as "keyword" type.

2. Step 2: Apply rule to field "username"

   Since "username" is a string field, it will be mapped as "keyword" according to the template.

3. Final Answer:

   "keyword" -> Option C

4. Quick Check:

   String fields mapped as keyword = "keyword" [OK]

Hint: match_mapping_type: string maps to keyword here [OK]

Common Mistakes:

• Assuming default type is text instead of keyword
• Confusing match and match_mapping_type
• Thinking string type is valid mapping in Elasticsearch 7+

4. You wrote this dynamic template but it does not apply to fields ending with _id. What is the error?

{
  "dynamic_templates": [
    {
      "ids": {
        "match": "*_id",
        "mapping": {
          "type": "keyword"
        }
      }
    }
  ]
}

medium

A. Incorrect mapping type, should be "text" instead of "keyword"

B. Dynamic templates must be outside the mappings block

C. Field name should be "path_match" not "match"

D. Missing "match_pattern": "regex" to enable wildcard matching

Solution

1. Step 1: Identify the matching issue

   The pattern "*_id" uses a wildcard, which requires "match_pattern": "regex" or "wildcard" to work properly.

2. Step 2: Confirm mapping and placement

   The mapping type "keyword" is correct for IDs, and dynamic templates are correctly placed inside mappings.

3. Final Answer:

   Missing "match_pattern": "regex" to enable wildcard matching -> Option D

4. Quick Check:

   Wildcard match needs match_pattern [OK]

Hint: Add match_pattern for wildcard or regex matches [OK]

Common Mistakes:

• Forgetting match_pattern with wildcards
• Changing mapping type unnecessarily
• Placing dynamic templates outside mappings

5. You want to create a dynamic template that maps all numeric fields with names containing _count as long type, but only if the field value is greater than zero. Which approach correctly applies this logic?

hard

A. Dynamic templates cannot filter by field value; use mapping for "long" and filter in queries

B. Use a dynamic template with match for "*_count" and mapping type "long"; add a runtime field to check > 0

C. Use a dynamic template with match for "*_count" and mapping type "long"; then filter documents with a query for values > 0

D. Use a dynamic template with match for "*_count" and mapping type "long" with a condition for > 0

Solution

1. Step 1: Understand dynamic templates capabilities

   Dynamic templates define field mappings based on name or type patterns, but cannot apply conditions based on field values.

2. Step 2: Identify correct approach for value filtering

   Filtering by field value (> 0) must be done at query time, not mapping time.

3. Final Answer:

   Dynamic templates cannot filter by field value; use mapping for "long" and filter in queries -> Option A

4. Quick Check:

   Value filtering = query, not mapping [OK]

Hint: Dynamic templates map fields; value filters happen in queries [OK]

Common Mistakes:

• Trying to filter field values in dynamic templates
• Adding unsupported 'condition' in mapping
• Confusing mapping with query filtering