Cluster, node, and shard architecture in Elasticsearch - Step-by-Step Execution

Concept Flow - Cluster, node, and shard architecture

Start: Client sends data/query

↓

Cluster receives request

↓

Cluster routes to Node

↓

Node manages Shards

↓

Primary Shard

↓

Store data

↓

Node responds to Cluster

↓

Cluster sends response to Client

The cluster receives requests and routes them to nodes, which manage shards (primary and replicas) to store and retrieve data.

Execution Sample

Elasticsearch

Cluster -> Node -> Primary Shard -> Store Data
Cluster -> Node -> Replica Shard -> Store Copy

Shows how a cluster routes data storage to nodes and their shards.

Execution Table

Step	Component	Action	Result
1	Client	Sends data or query	Request sent to cluster
2	Cluster	Receives request	Routes to appropriate node
3	Node	Receives routed request	Determines shard to use
4	Primary Shard	Stores data or processes query	Data saved or query executed
5	Replica Shard	Stores copy of data	Backup created for fault tolerance
6	Node	Aggregates shard results	Prepares response
7	Cluster	Collects node responses	Forms final response
8	Client	Receives response	Operation complete

💡 Client receives response, ending the request cycle

Variable Tracker

Component	Start	After Step 2	After Step 4	After Step 5	Final
Cluster	Idle	Received request	Waiting for node	Waiting for node	Sent response
Node	Idle	Received request	Processed primary shard	Processed replica shard	Sent data to cluster
Primary Shard	Empty	N/A	Stored data	N/A	N/A
Replica Shard	Empty	N/A	N/A	Stored copy	N/A

Key Moments - 3 Insights

Why does the cluster route requests to nodes instead of handling data directly?

The cluster manages the overall system but nodes hold the actual data shards; routing to nodes allows distributed storage and processing, as shown in execution_table steps 2 and 3.

What is the difference between a primary shard and a replica shard?

Primary shards store the original data and handle write operations, while replica shards store copies for backup and fault tolerance, as seen in steps 4 and 5.

How does the system ensure data is not lost if a node fails?

Replica shards on other nodes keep copies of data, so if a node fails, the cluster can use replicas to maintain data availability, explained in step 5.

Visual Quiz - 3 Questions

Look at the execution_table, at which step does the primary shard store data?

A Step 4

B Step 3

C Step 5

D Step 2

💡 Hint

Check the 'Action' column for 'Stores data or processes query' in execution_table row 4

According to variable_tracker, what is the state of the replica shard after Step 5?

A Empty

B Processed primary shard

C Stored copy

D Sent data to cluster

💡 Hint

Look at the 'Replica Shard' row under 'After Step 5' in variable_tracker

If the cluster did not route requests to nodes, what would change in the execution_table?

A Step 1 would be repeated

B Steps 3 to 7 would be missing

C Step 8 would happen earlier

D Step 2 would be repeated

💡 Hint

Routing to nodes is shown in steps 3 to 7 in execution_table

Concept Snapshot

Cluster manages the whole system.
Nodes hold data shards.
Shards split data: primary (main) and replica (backup).
Cluster routes requests to nodes.
Nodes handle data storage and queries.
Replicas ensure data safety if nodes fail.

Full Transcript

In Elasticsearch, a cluster is a group of nodes working together. When a client sends data or a query, the cluster receives it and routes it to the right node. Each node manages shards, which are pieces of the data. Primary shards store the original data and handle writes. Replica shards keep copies for backup and fault tolerance. The node processes the request using these shards and sends the result back to the cluster, which then responds to the client. This system helps store data safely and handle queries efficiently.