Index settings (shards, replicas) in Elasticsearch - Step-by-Step Execution

Concept Flow - Index settings (shards, replicas)

Create Index Request

↓

Set number_of_shards

↓

Set number_of_replicas

↓

Elasticsearch Distributes Shards

↓

Index Ready for Use

When creating an index, you set how many shards and replicas it has. Elasticsearch then distributes these shards across nodes.

Execution Sample

Elasticsearch

{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 2
  }
}

This JSON sets an index with 3 primary shards and 2 replicas for each shard.

Execution Table

Step	Action	Setting Changed	Value	Effect
1	Create index request received	none	none	Start index creation
2	Set number_of_shards	number_of_shards	3	Index will have 3 primary shards
3	Set number_of_replicas	number_of_replicas	2	Each primary shard will have 2 replicas
4	Elasticsearch distributes shards	shard allocation	3 primaries + 6 replicas	Shards spread across nodes
5	Index ready	none	none	Index can accept data and queries

💡 Index creation completes after shards and replicas are allocated

Variable Tracker

Variable	Start	After Step 2	After Step 3	Final
number_of_shards	undefined	3	3	3
number_of_replicas	undefined	undefined	2	2
total_shards	undefined	undefined	undefined	9 (3 primaries + 6 replicas)

Key Moments - 2 Insights

Why do we set number_of_shards before number_of_replicas?

Because replicas are copies of primary shards, you must first decide how many primary shards exist (see execution_table steps 2 and 3).

What happens if number_of_replicas is set to 0?

No replicas are created, so data is only on primary shards. This reduces fault tolerance (not shown in this trace but implied after step 3).

Visual Quiz - 3 Questions

Look at the execution_table, what is the value of number_of_shards after step 2?

A 3

B 2

C undefined

D 9

💡 Hint

Check the 'Setting Changed' and 'Value' columns at step 2 in execution_table

At which step does Elasticsearch distribute shards across nodes?

A Step 2

B Step 3

C Step 4

D Step 5

💡 Hint

Look for 'Elasticsearch distributes shards' in the Action column of execution_table

If number_of_replicas was set to 1 instead of 2, what would be the total shards allocated?

A 3

B 6 (3 primaries + 3 replicas)

C 6

D 9

💡 Hint

Refer to variable_tracker's total_shards calculation and adjust replicas accordingly

Concept Snapshot

Index settings control data distribution.
number_of_shards sets primary pieces.
number_of_replicas sets copies for fault tolerance.
Total shards = primaries + replicas.
Set shards first, then replicas.
Elasticsearch spreads shards across nodes.

Full Transcript

When creating an Elasticsearch index, you specify settings for number_of_shards and number_of_replicas. First, the number_of_shards is set, which determines how many primary pieces the data is split into. Then, number_of_replicas is set, which defines how many copies of each primary shard exist for safety. Elasticsearch then distributes these shards and replicas across the cluster nodes. The total shards equal the sum of primary shards and their replicas. This process ensures data is balanced and fault tolerant.

Practice

1. What is the main purpose of setting number_of_shards in an Elasticsearch index?

easy

A. To define the size of the index

B. To create backup copies of data

C. To split data into parts for faster searching

D. To set the maximum number of documents

Solution

1. Step 1: Understand shards role

   Shards divide the index data into smaller parts to allow parallel searching.

2. Step 2: Differentiate shards from replicas

   Replicas are copies for safety, not for splitting data.

3. Final Answer:

   To split data into parts for faster searching -> Option C

4. Quick Check:

   Shards = split data for speed [OK]

Hint: Shards split data, replicas copy data [OK]

Common Mistakes:

• Confusing shards with replicas
• Thinking shards create backups
• Assuming shards set index size

2. Which of the following is the correct way to set 3 shards and 2 replicas in an Elasticsearch index creation JSON?

easy

A. { "settings": { "number_of_shards": 3, "number_of_replicas": 2 } }

B. { "settings": { "shards": 3, "replicas": 2 } }

C. { "settings": { "shard_count": 3, "replica_count": 2 } }

D. { "settings": { "num_shards": 3, "num_replicas": 2 } }

Solution

1. Step 1: Recall correct setting keys

   The official keys are number_of_shards and number_of_replicas.

2. Step 2: Match keys with options

   Only { "settings": { "number_of_shards": 3, "number_of_replicas": 2 } } uses the exact correct keys and valid JSON format.

3. Final Answer:

   { "settings": { "number_of_shards": 3, "number_of_replicas": 2 } } -> Option A

4. Quick Check:

   Correct keys = number_of_shards, number_of_replicas [OK]

Hint: Use full keys: number_of_shards and number_of_replicas [OK]

Common Mistakes:

• Using shorthand keys like shards or replicas
• Wrong JSON syntax
• Mixing shard and replica counts

3. Given an index with number_of_shards set to 4 and number_of_replicas set to 1, how many total shard copies exist in the cluster?

medium

A. 4

B. 6

C. 5

D. 8

Solution

1. Step 1: Calculate primary shards

   There are 4 primary shards as set.

2. Step 2: Calculate replica shards

   Each primary shard has 1 replica, so 4 replicas total.

3. Step 3: Add primary and replicas

   Total shards = 4 primary + 4 replicas = 8 shards.

4. Final Answer:

   8 -> Option D

5. Quick Check:

   Total shards = primary + replicas = 8 [OK]

Hint: Total shards = shards x (1 + replicas) [OK]

Common Mistakes:

• Counting only primary shards
• Adding shards and replicas incorrectly
• Confusing replicas with shards

4. You tried to create an index with this setting: { "settings": { "number_of_shards": 2, "number_of_replicas": -1 } }. What is the problem?

medium

A. Negative replicas value is invalid

B. Number of shards cannot be 2

C. Missing required field for replicas

D. JSON syntax error

Solution

1. Step 1: Check shards value

   Number of shards is 2, which is valid.

2. Step 2: Check replicas value

   Replicas cannot be negative; -1 is invalid.

3. Final Answer:

   Negative replicas value is invalid -> Option A

4. Quick Check:

   Replicas must be zero or positive [OK]

Hint: Replicas cannot be negative numbers [OK]

Common Mistakes:

• Assuming negative replicas mean no replicas
• Thinking shards must be > 3
• Ignoring JSON correctness

5. You want to create an index optimized for high availability with 5 shards. Which setting combination is best to ensure data safety and fast search?

hard

A. { "number_of_shards": 5, "number_of_replicas": 0 }

B. { "number_of_shards": 5, "number_of_replicas": 1 }

C. { "number_of_shards": 1, "number_of_replicas": 5 }

D. { "number_of_shards": 10, "number_of_replicas": 1 }

Solution

1. Step 1: Understand shards and replicas roles

   Shards split data for speed; replicas provide copies for safety.

2. Step 2: Evaluate options for availability and speed

   { "number_of_shards": 5, "number_of_replicas": 1 } keeps 5 shards for speed and 1 replica for safety, balancing both.

3. Step 3: Reject other options

   The option with zero replicas has no safety, the option with 1 shard is slow for search, the option with 10 shards has overhead.

4. Final Answer:

   { "number_of_shards": 5, "number_of_replicas": 1 } -> Option B

5. Quick Check:

   Balance shards and replicas for speed and safety [OK]

Hint: Balance shards for speed and replicas for safety [OK]

Common Mistakes:

• Setting zero replicas for safety
• Using too few shards for speed
• Over-sharding causing overhead