Why templates standardize index creation in Elasticsearch - See It in Action

Why templates standardize index creation

📖 Scenario: You work in a company that collects logs from many servers. Each day, a new index is created to store that day's logs. To keep the data organized and consistent, you want to use a template that sets the rules for all daily log indexes.

🎯 Goal: Create an Elasticsearch index template that standardizes the creation of daily log indexes with consistent settings and mappings.

📋 What You'll Learn

Create an index template named daily_logs_template

Set the template to apply to index names starting with logs-

Define settings with number_of_shards set to 1

Define mappings with a timestamp field of type date

Define mappings with a message field of type text

💡 Why This Matters

🌍 Real World

Companies use index templates to manage large volumes of log or event data that arrive daily, ensuring all indexes have the same structure and performance settings.

💼 Career

Knowing how to create and use index templates is important for roles like DevOps engineers, data engineers, and anyone managing Elasticsearch clusters to maintain data consistency and optimize search performance.

1

Create the basic index template structure

Create a variable called daily_logs_template and assign it a dictionary with keys index_patterns set to a list containing "logs-*", and an empty template dictionary.

Elasticsearch

daily_logs_template = {
    "index_patterns": ["logs-*"] ,
    "template": {}
}
# Your code here

Hint

Start by creating a dictionary with the keys index_patterns and template.

2

Add settings to the template

Inside the template dictionary of daily_logs_template, add a settings dictionary with number_of_shards set to 1.

Elasticsearch

daily_logs_template = {
    "index_patterns": ["logs-*"] ,
    "template": {
        "settings": {
            # Your code here
        }
    }
}

Hint

Settings control how the index stores data. Use number_of_shards to set the number of pieces the index is split into.

3

Add mappings for fields

Inside the template dictionary of daily_logs_template, add a mappings dictionary with properties defining a timestamp field of type date and a message field of type text.

Elasticsearch

daily_logs_template = {
    "index_patterns": ["logs-*"] ,
    "template": {
        "settings": {
            "number_of_shards": 1
        },
        "mappings": {
            "properties": {
                # Your code here
            }
        }
    }
}

Hint

Mappings define the data types of fields in the index. Use properties to list fields and their types.

4

Complete the template for use

Ensure the daily_logs_template dictionary includes index_patterns, template with settings and mappings as defined, ready to be used to create indexes.

Elasticsearch

daily_logs_template = {
    "index_patterns": ["logs-*"] ,
    "template": {
        "settings": {
            "number_of_shards": 1
        },
        "mappings": {
            "properties": {
                "timestamp": {"type": "date"},
                "message": {"type": "text"}
            }
        }
    }
}
# Your code here

Hint

Review the full template dictionary to confirm all parts are included.

Practice

1. Why do Elasticsearch index templates help when creating multiple indexes with similar data?

easy

A. They speed up the search queries by caching results.

B. They delete old indexes to save space.

C. They automatically apply consistent settings and mappings to all matching indexes.

D. They convert data into a different format before indexing.

Solution

1. Step 1: Understand the purpose of index templates

   Index templates define settings and mappings that apply automatically to new indexes matching a pattern.

2. Step 2: Recognize the benefit of consistency

   Using templates ensures all similar indexes have the same structure, avoiding manual errors and saving time.

3. Final Answer:

   They automatically apply consistent settings and mappings to all matching indexes. -> Option C

4. Quick Check:

   Templates = consistent index setup [OK]

Hint: Templates set index rules automatically for matching names [OK]

Common Mistakes:

• Thinking templates speed up searches directly
• Confusing templates with index deletion tools
• Believing templates change data format

2. Which of the following is the correct way to define an index template in Elasticsearch?

easy

A. { "template": { "shards": 1 }, "pattern": ["logs"] }

B. { "index": { "shards": 1 }, "pattern": "logs-*" }

C. { "settings": { "shards": 1 }, "match": "logs-*" }

D. { "template": { "settings": { "number_of_shards": 1 } }, "index_patterns": ["logs-*"] }

Solution

1. Step 1: Recall the correct template structure

   Elasticsearch templates use "index_patterns" to match index names and "template" to define settings and mappings.

2. Step 2: Identify the valid JSON format

   { "template": { "settings": { "number_of_shards": 1 } }, "index_patterns": ["logs-*"] } correctly uses "index_patterns" and nests settings inside "template".

3. Final Answer:

   { "template": { "settings": { "number_of_shards": 1 } }, "index_patterns": ["logs-*"] } -> Option D

4. Quick Check:

   Correct template keys = { "template": { "settings": { "number_of_shards": 1 } }, "index_patterns": ["logs-*"] } [OK]

Hint: Look for "index_patterns" and nested "template" keys [OK]

Common Mistakes:

• Using "pattern" instead of "index_patterns"
• Placing settings outside the "template" object
• Using incorrect JSON keys like "match"

3. Given this template:

{
  "index_patterns": ["sales-*"],
  "template": {
    "settings": { "number_of_replicas": 2 },
    "mappings": { "properties": { "price": { "type": "float" } } }
  }
}

What will be the number_of_replicas for an index named sales-2024 created after this template?

medium

A. 2

B. 1

C. 0

D. 3

Solution

1. Step 1: Match index name with template pattern

   The index name "sales-2024" matches the pattern "sales-*" in the template.

2. Step 2: Apply template settings to the index

   The template sets "number_of_replicas" to 2, so the new index will have 2 replicas.

3. Final Answer:

   2 -> Option A

4. Quick Check:

   Matching pattern applies replicas = 2 [OK]

Hint: Match index name to pattern, then apply template settings [OK]

Common Mistakes:

• Assuming default replicas instead of template value
• Ignoring pattern matching rules
• Confusing replicas with shards

4. You created this template:

{
  "index_patterns": ["app-*"],
  "template": {
    "settings": { "number_of_shards": 3 }
  }
}

But new indexes named app-logs are created with 5 shards instead of 3. What is the likely problem?

medium

A. The template was created after the indexes were created.

B. The index name does not match the pattern "app-*".

C. The setting "number_of_shards" cannot be set by templates.

D. The template JSON syntax is invalid.

Solution

1. Step 1: Understand when templates apply

   Templates apply only when an index is created. Existing indexes keep their original settings.

2. Step 2: Check index creation timing

   If indexes were created before the template, they won't have the template's shard settings.

3. Final Answer:

   The template was created after the indexes were created. -> Option A

4. Quick Check:

   Templates affect new indexes only [OK]

Hint: Templates apply only at index creation time [OK]

Common Mistakes:

• Thinking templates update existing indexes
• Assuming pattern mismatch without checking
• Believing shard count can't be set in templates

5. You want to ensure all indexes starting with user-data- have the same mapping for a username field as keyword type and 1 shard. Which template JSON correctly achieves this?

hard

A. { "index_patterns": ["user-data-*"], "template": { "settings": { "number_of_replicas": 1 }, "mappings": { "properties": { "username": { "type": "keyword" } } } } }

B. { "index_patterns": ["user-data-*"], "template": { "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "username": { "type": "keyword" } } } } }

C. { "index_patterns": ["user-data-*"], "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "username": { "type": "text" } } } }

D. { "pattern": "user-data-*", "settings": { "shards": 1 }, "mappings": { "username": "keyword" } }

Solution

1. Step 1: Check index pattern and template structure

   { "index_patterns": ["user-data-*"], "template": { "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "username": { "type": "keyword" } } } } } uses "index_patterns" and nests settings and mappings inside "template", which is correct.

2. Step 2: Verify settings and mapping details

   It sets "number_of_shards" to 1 and maps "username" as "keyword", matching requirements.

3. Final Answer:

   { "index_patterns": ["user-data-*"], "template": { "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "username": { "type": "keyword" } } } } } -> Option B

4. Quick Check:

   Correct pattern, shards, and mapping = { "index_patterns": ["user-data-*"], "template": { "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "username": { "type": "keyword" } } } } } [OK]

Hint: Use "index_patterns" and nest settings/mappings inside "template" [OK]

Common Mistakes:

• Using wrong keys like "pattern" or placing settings outside "template"
• Setting wrong shard or replica counts
• Mapping "username" as "text" instead of "keyword"