Index templates in Elasticsearch - Mini Project: Build & Apply

Create and Use an Elasticsearch Index Template

📖 Scenario: You are managing a logging system that stores logs from multiple applications. To keep the data organized and consistent, you want to create an index template in Elasticsearch. This template will automatically apply settings and mappings to new log indexes that match a specific pattern.

🎯 Goal: Build an Elasticsearch index template named log_template that applies to indexes starting with logs-. The template should set the number of shards to 1 and define a mapping for a timestamp field as a date type.

📋 What You'll Learn

Create an index template named log_template

Set the index pattern to logs-*

Configure the template to set number_of_shards to 1

Add a mapping for the timestamp field with type date

💡 Why This Matters

🌍 Real World

Index templates help automate consistent settings and mappings for new indexes, saving time and avoiding errors in large logging or data systems.

💼 Career

Understanding index templates is essential for roles like DevOps, data engineers, and backend developers working with Elasticsearch to manage scalable search and analytics systems.

1

Create the basic index template structure

Create an index template named log_template with an empty index_patterns list and empty template object.

Elasticsearch

{
  "index_patterns": [],
  "template": {}
  # Your code here
}

Hint

Use "index_patterns": ["logs-*"] to match indexes starting with logs-.

2

Add index settings for shards

Inside the template object, add settings with number_of_shards set to 1.

Elasticsearch

{
  "index_patterns": ["logs-*"] ,
  "template": {
    "settings": {
      # Your code here
    }
  }
}

Hint

Put "number_of_shards": 1 inside settings.

3

Add mapping for the timestamp field

Inside the template object, add a mappings object with a properties field. Define timestamp as a date type.

Elasticsearch

{
  "index_patterns": ["logs-*"] ,
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "mappings": {
      "properties": {
        # Your code here
      }
    }
  }
}

Hint

Define timestamp inside properties with "type": "date".

4

Complete the index template JSON

Ensure the full JSON includes index_patterns, template with settings and mappings as specified. This is the final complete index template.

Elasticsearch

{
  "index_patterns": ["logs-*"] ,
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "mappings": {
      "properties": {
        "timestamp": {
          "type": "date"
        }
      }
    }
  }
}
# Your code here to complete the template if needed

Hint

Review the entire JSON to confirm all parts are included and correctly formatted.

Practice

1.

What is the main purpose of an index template in Elasticsearch?

easy

A. To delete old indexes automatically

B. To run queries faster on existing indexes

C. To backup data from indexes

D. To automatically apply settings and mappings to new indexes matching a pattern

Solution

1. Step 1: Understand what index templates do

   Index templates define settings and mappings that apply automatically to new indexes matching a pattern.

2. Step 2: Compare options with this purpose

   Only To automatically apply settings and mappings to new indexes matching a pattern describes this automatic application of settings and mappings.

3. Final Answer:

   To automatically apply settings and mappings to new indexes matching a pattern -> Option D

4. Quick Check:

   Index templates = auto apply settings [OK]

Hint: Index templates set rules for new indexes automatically [OK]

Common Mistakes:

• Thinking templates delete or backup data
• Confusing templates with query optimization
• Believing templates apply to existing indexes only

2.

Which of the following is the correct way to specify an index pattern in an Elasticsearch index template?

{
  "index_patterns": ["logs-*"]
}

easy

A. "index_patterns": "logs-*"

B. "index_patterns": ["logs-*"]

C. "index_patterns": {"logs-*"}

D. "index_patterns": logs-*

Solution

1. Step 1: Recall correct JSON syntax for index_patterns

   The index_patterns field expects an array of strings, so it must be inside square brackets.

2. Step 2: Check each option's syntax

   Only "index_patterns": ["logs-*"] uses an array with a string inside: ["logs-*"] which is correct JSON syntax.

3. Final Answer:

   "index_patterns": ["logs-*"] -> Option B

4. Quick Check:

   Patterns must be in array format [OK]

Hint: Patterns always go inside square brackets as a list [OK]

Common Mistakes:

• Using a string instead of an array
• Using curly braces instead of square brackets
• Omitting quotes around the pattern

3.

Given this index template:

{
  "index_patterns": ["app-logs-*"],
  "template": {
    "settings": {"number_of_shards": 2},
    "mappings": {"properties": {"user": {"type": "keyword"}}}
  }
}

What will be the number of shards and mapping type for the index app-logs-2024?

medium

A. 2 shards, user field as text

B. 1 shard, user field as text

C. 2 shards, user field as keyword

D. 1 shard, user field as keyword

Solution

1. Step 1: Match index name with pattern

   The index name app-logs-2024 matches the pattern app-logs-* in the template.

2. Step 2: Apply template settings and mappings

   The template sets number_of_shards to 2 and maps the user field as keyword type.

3. Final Answer:

   2 shards, user field as keyword -> Option C

4. Quick Check:

   Matching pattern applies template settings [OK]

Hint: Check if index name fits pattern, then apply template settings [OK]

Common Mistakes:

• Assuming default shard count instead of template value
• Confusing keyword with text mapping type
• Ignoring pattern matching rules

4.

Identify the error in this index template snippet:

{
  "index_patterns": "logs-*",
  "template": {
    "settings": {"number_of_replicas": 1}
  }
}

medium

A. "index_patterns" should be an array, not a string

B. "number_of_replicas" is not a valid setting

C. "template" key is missing required "mappings"

D. The pattern "logs-*" is invalid

Solution

1. Step 1: Check the type of index_patterns

   The index_patterns field must be an array of strings, but here it is a single string.

2. Step 2: Validate other fields

   number_of_replicas is a valid setting, mappings is optional, and the pattern syntax is correct.

3. Final Answer:

   "index_patterns" should be an array, not a string -> Option A

4. Quick Check:

   index_patterns must be an array [OK]

Hint: index_patterns always needs square brackets [OK]

Common Mistakes:

• Using string instead of array for index_patterns
• Thinking mappings are always required
• Assuming pattern syntax is wrong when it's correct

5.

You want to create an index template that applies to all indexes starting with sales- but excludes those ending with -archive. Which approach correctly achieves this?

hard

A. Use index_patterns": ["sales-*"] and create a separate template with higher priority for sales-*-archive that overrides settings

B. Use index_patterns": ["sales-*"] and rely on manual exclusion when creating indexes

C. Use index_patterns": ["sales-*"] and add a composed_of template excluding sales-*-archive

D. Use index_patterns": ["sales-*"] and add a priority template for sales-*-archive with no settings

Solution

1. Step 1: Understand template priority and pattern matching

   Elasticsearch applies templates by priority; higher priority templates override lower ones.

2. Step 2: Exclude indexes by overriding with higher priority template

   To exclude sales-*-archive, create a separate template with higher priority that overrides or clears settings for those indexes.

3. Final Answer:

   Create a higher priority template for sales-*-archive to override settings -> Option A

4. Quick Check:

   Use priority to override unwanted index templates [OK]

Hint: Use higher priority template to override exclusions [OK]

Common Mistakes:

• Assuming manual exclusion works automatically
• Using composed_of incorrectly for exclusion
• Not setting template priority to control overrides