Template priority and composition in Elasticsearch - Mini Project: Build & Apply

Template Priority and Composition in Elasticsearch

📖 Scenario: You are managing an Elasticsearch cluster that uses index templates to define settings and mappings for new indexes. You want to learn how to create multiple templates with different priorities and how Elasticsearch composes them when creating an index.

🎯 Goal: Build multiple index templates with different priorities and overlapping index patterns. Learn how Elasticsearch merges these templates based on priority and composition rules.

📋 What You'll Learn

Create two index templates with overlapping index patterns

Assign different priorities to each template

Use template composition with composed_of to reuse component templates

Create a new index that matches the templates

Observe the final merged settings and mappings applied to the index

💡 Why This Matters

🌍 Real World

Index templates help manage settings and mappings for many indexes automatically, saving time and avoiding errors.

💼 Career

Understanding template priority and composition is important for Elasticsearch administrators and developers managing scalable search systems.

1

Create a component template called base_settings

Create a component template called base_settings with the following settings: number_of_shards set to 1 and number_of_replicas set to 1.

Elasticsearch

# Create component template base_settings with number_of_shards=1 and number_of_replicas=1
# Your code here

Hint

Use the PUT _component_template/base_settings API with a JSON body containing settings for shards and replicas.

2

Create two index templates with different priorities and composition

Create two index templates: template_low with priority 10 and template_high with priority 20. Both templates should apply to index pattern logs-*. template_low should compose the base_settings component template and add a mapping for a message field of type text. template_high should add a mapping for a user field of type keyword.

Elasticsearch

PUT _component_template/base_settings
{
  "template": {
    "settings": {
      "number_of_shards": "1",
      "number_of_replicas": "1"
    }
  }
}

# Create template_low with priority 10, compose base_settings, add message mapping
# Create template_high with priority 20, add user mapping
# Your code here

Hint

Use PUT _index_template/ API to create each template. Use priority and composed_of fields as described.

3

Create an index that matches the templates

Create a new index called logs-2024-06 that matches the index pattern logs-* to apply the templates.

Elasticsearch

PUT _component_template/base_settings
{
  "template": {
    "settings": {
      "number_of_shards": "1",
      "number_of_replicas": "1"
    }
  }
}

PUT _index_template/template_low
{
  "index_patterns": ["logs-*"] ,
  "priority": 10,
  "composed_of": ["base_settings"],
  "template": {
    "mappings": {
      "properties": {
        "message": { "type": "text" }
      }
    }
  }
}

PUT _index_template/template_high
{
  "index_patterns": ["logs-*"] ,
  "priority": 20,
  "template": {
    "mappings": {
      "properties": {
        "user": { "type": "keyword" }
      }
    }
  }
}

# Create the index logs-2024-06
# Your code here

Hint

Use the PUT API with the index name logs-2024-06 to create the index.

4

Check the final merged mappings and settings of the index

Use the GET logs-2024-06/_mapping and GET logs-2024-06/_settings APIs to print the final mappings and settings applied to the logs-2024-06 index.

Elasticsearch

PUT _component_template/base_settings
{
  "template": {
    "settings": {
      "number_of_shards": "1",
      "number_of_replicas": "1"
    }
  }
}

PUT _index_template/template_low
{
  "index_patterns": ["logs-*"] ,
  "priority": 10,
  "composed_of": ["base_settings"],
  "template": {
    "mappings": {
      "properties": {
        "message": { "type": "text" }
      }
    }
  }
}

PUT _index_template/template_high
{
  "index_patterns": ["logs-*"] ,
  "priority": 20,
  "template": {
    "mappings": {
      "properties": {
        "user": { "type": "keyword" }
      }
    }
  }
}

PUT logs-2024-06

# Get and print the mappings and settings of logs-2024-06
# Your code here

Hint

Use GET logs-2024-06/_mapping and GET logs-2024-06/_settings to see the final applied templates.

Practice

1. What does the priority setting control in Elasticsearch index templates?

easy

A. Which template applies first when multiple templates match an index

B. The order of fields in the index mapping

C. The number of shards for the index

D. The refresh interval of the index

Solution

1. Step 1: Understand the role of priority in templates

   The priority setting determines which template Elasticsearch applies first when multiple templates match the same index.

2. Step 2: Differentiate priority from other settings

   Priority does not affect shard count, field order, or refresh interval; it only controls template application order.

3. Final Answer:

   Which template applies first when multiple templates match an index -> Option A

4. Quick Check:

   Priority controls template application order = B [OK]

Hint: Priority decides template order when multiple match [OK]

Common Mistakes:

• Confusing priority with shard settings
• Thinking priority affects field order
• Assuming priority controls refresh interval

2. Which of the following is the correct way to define a template with priority 10 in Elasticsearch?

easy

A. { "index_patterns": ["logs-*"], "priority": true, "template": { "settings": {} } }

B. { "index_patterns": ["logs-*"], "priority": "high", "template": { "settings": {} } }

C. { "index_patterns": ["logs-*"], "priority": 10, "template": { "settings": {} } }

D. { "index_patterns": ["logs-*"], "priority": null, "template": { "settings": {} } }

Solution

1. Step 1: Check the priority value type

   Priority must be a numeric value, such as 10, not a string or boolean.

2. Step 2: Validate JSON structure

   { "index_patterns": ["logs-*"], "priority": 10, "template": { "settings": {} } } correctly uses an integer 10 for priority and valid JSON structure for template.

3. Final Answer:

   { "index_patterns": ["logs-*"], "priority": 10, "template": { "settings": {} } } -> Option C

4. Quick Check:

   Priority is a number, not string or boolean = C [OK]

Hint: Priority must be a number, not string or boolean [OK]

Common Mistakes:

• Using string like "high" instead of number
• Setting priority to true or null
• Incorrect JSON structure for template

3. Given two templates: Template A with priority 5 and Template B with priority 10, both matching index logs-2024. Which template settings will Elasticsearch apply first?

medium

A. Template B settings will apply first

B. Elasticsearch throws an error due to conflict

C. Both templates apply simultaneously with no order

D. Template A settings will apply first

Solution

1. Step 1: Understand priority order

   Elasticsearch sorts matching templates by priority in ascending order (lowest first) and merges their configurations, with later (higher priority) templates overriding earlier ones.

2. Step 2: Compare priorities of Template A and B

   Template A has priority 5, which is lower than Template B's 10, so Template A applies first.

3. Final Answer:

   Template A settings will apply first -> Option D

4. Quick Check:

   Lower priority number applies first = B [OK]

Hint: Lower priority number means earlier application [OK]

Common Mistakes:

• Thinking higher priority applies first
• Thinking templates apply simultaneously
• Expecting error on priority conflict

4. You have two templates: Template X with composed_of referencing Template Y, but Template Y is missing. What will happen when you try to create an index matching Template X?

medium

A. Template X applies without Template Y's settings

B. Index creation fails with an error about missing template

C. Elasticsearch ignores Template X and uses default settings

D. Template Y is automatically created as empty

Solution

1. Step 1: Understand template composition with composed_of

   Templates using composed_of require referenced templates to exist.

2. Step 2: Check behavior when referenced template is missing

   If Template Y is missing, Elasticsearch throws an error during index creation because it cannot compose Template X fully.

3. Final Answer:

   Index creation fails with an error about missing template -> Option B

4. Quick Check:

   Missing composed_of template causes error = A [OK]

Hint: All composed_of templates must exist or error occurs [OK]

Common Mistakes:

• Assuming missing templates are ignored
• Expecting automatic creation of missing templates
• Thinking index creation succeeds without all templates

5. You want to create a flexible index template setup where a base template defines common settings, and multiple specialized templates add mappings for different data types. How should you use priority and composition to achieve this?

hard

A. Create a base template with low priority and use composed_of in specialized templates with higher priority

B. Create specialized templates with low priority and base template with highest priority without composition

C. Use only one template with all settings combined, no priority or composition needed

D. Set all templates with the same priority and avoid composition

Solution

1. Step 1: Use base template for common settings with low priority

   The base template should have a lower priority so it is merged first in the composition order, providing common settings that higher priority templates can override.

2. Step 2: Use composed_of in specialized templates with higher priority

   Specialized templates compose the base template and have higher priority to add or override mappings.

3. Step 3: Combine priority and composition for flexible layering

   This setup allows reusable base settings and flexible specialized additions.

4. Final Answer:

   Create a base template with low priority and use composed_of in specialized templates with higher priority -> Option A

5. Quick Check:

   Base low priority + composed_of in higher priority = D [OK]

Hint: Base template low priority, specialized templates compose with higher priority [OK]

Common Mistakes:

• Setting base template with highest priority
• Not using composition for reusable settings
• Using same priority for all templates