Why Elasticsearch exists - Visual Breakdown

Concept Flow - Why Elasticsearch exists

Need to search large data

↓

Traditional DB too slow

↓

Need fast, scalable search

↓

Elasticsearch created

↓

Distributed, real-time search engine

↓

Fast, scalable, easy to use

Shows the flow from the need to search large data quickly to the creation of Elasticsearch as a fast, scalable search engine.

Execution Sample

Elasticsearch

GET /_search
{
  "query": {
    "match": { "message": "search text" }
  }
}

This query searches documents containing 'search text' in the 'message' field using Elasticsearch.

Execution Table

Step	Action	Input	Process	Output
1	Receive search request	{"query":{"match":{"message":"search text"}}}	Parse JSON query	Parsed query object
2	Distribute query	Parsed query object	Send to all relevant shards	Queries sent to shards
3	Search shards	Query on each shard	Each shard searches its data	Partial search results
4	Aggregate results	Partial search results	Combine and sort results	Final sorted search results
5	Return results	Final sorted search results	Send back to client	Search results displayed

💡 Search results returned to client, fulfilling the fast and scalable search need.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
query	null	{"match":{"message":"search text"}}	Distributed to shards	Executed on shards	Partial results collected	Final sorted results

Key Moments - 2 Insights

Why can't traditional databases handle large search queries quickly?

Traditional databases are designed for structured data and transactions, not for full-text search at scale. As shown in execution_table step 1 and 2, Elasticsearch distributes queries to shards to handle large data efficiently.

How does Elasticsearch achieve fast search results?

Elasticsearch splits data into shards and searches them in parallel (step 3), then aggregates results quickly (step 4), enabling fast responses even with big data.

Visual Quiz - 3 Questions

Look at the execution_table, what happens at step 3?

A The query is parsed from JSON

B Results are aggregated and sorted

C Each shard searches its own data

D Results are returned to the client

💡 Hint

Check the 'Process' column at step 3 in execution_table.

According to variable_tracker, what is the state of 'query' after step 4?

A Partial results collected

B Executed on shards

C Distributed to shards

D Final sorted results

💡 Hint

Look at the 'query' row and the 'After Step 4' column in variable_tracker.

If Elasticsearch did not distribute queries to shards, which step would be missing?

A Step 1: Receive search request

B Step 2: Distribute query

C Step 4: Aggregate results

D Step 5: Return results

💡 Hint

Refer to execution_table step 2 description.

Concept Snapshot

Why Elasticsearch exists:
- Need: Fast search on large data
- Traditional DBs too slow for full-text search
- Elasticsearch: distributed, real-time search engine
- Splits data into shards for parallel search
- Aggregates results quickly for fast response

Full Transcript

Elasticsearch was created because people needed a way to search large amounts of data quickly. Traditional databases are not designed for fast full-text search on big data. Elasticsearch solves this by splitting data into smaller parts called shards and searching them at the same time. It then combines the results and sends them back fast. This makes searching large data sets easy and quick.