Cluster, node, and shard architecture in Elasticsearch

Introduction

Elasticsearch stores and searches data efficiently by splitting work across many machines. It uses clusters, nodes, and shards to organize and manage data.

When you want to store large amounts of data that don't fit on one computer.

When you need fast search results by spreading work across multiple machines.

When you want your data to be safe even if one machine fails.

When you want to add more machines to handle more data or users.

When you want to organize data so it can be searched and updated quickly.

Syntax

Elasticsearch

Cluster: A group of one or more nodes working together.
Node: A single machine in the cluster that stores data and participates in search.
Shard: A piece of an index, holding part of the data.

Index: A collection of documents split into shards.

A cluster has many nodes.

An index is divided into shards for better performance and reliability.

Examples

This example shows a cluster with three nodes. The 'products' index is split into 5 parts (shards), and each shard has a copy (replica) for safety.

Elasticsearch

Cluster: my-elasticsearch-cluster
Nodes: node-1, node-2, node-3
Index: products
Shards: 5 primary shards + 1 replica shard each

Each node holds some shards and replicas. This spreads data and workload evenly.

Elasticsearch

Node: node-1
Stores: shard 1, shard 2, replica of shard 3

Node: node-2
Stores: shard 3, shard 4, replica of shard 1

Node: node-3
Stores: shard 5, replica of shard 2, replica of shard 4

Sample Program

These commands check the cluster health, list nodes, and show shard allocation for the 'products' index.

Elasticsearch

GET /_cluster/health
GET /_cat/nodes?v
GET /products/_shards

Important Notes

Primary shards hold the original data; replica shards are copies for backup and speed.

Nodes can have different roles, like master node or data node, but all work together.

Shards help Elasticsearch handle big data by splitting it into smaller parts.

Summary

A cluster is a group of nodes working together.

A node is a single machine in the cluster.

An index is split into shards to store data efficiently and safely.