Why Elasticsearch exists - Visual Breakdown

Concept Flow - Why Elasticsearch exists

Need to search large data

↓

Traditional DB too slow

↓

Need fast, scalable search

↓

Elasticsearch created

↓

Distributed, real-time search engine

↓

Fast, scalable, easy to use

Shows the flow from the need to search large data quickly to the creation of Elasticsearch as a fast, scalable search engine.

Execution Sample

Elasticsearch

GET /_search
{
  "query": {
    "match": { "message": "search text" }
  }
}

This query searches documents containing 'search text' in the 'message' field using Elasticsearch.

Execution Table

Step	Action	Input	Process	Output
1	Receive search request	{"query":{"match":{"message":"search text"}}}	Parse JSON query	Parsed query object
2	Distribute query	Parsed query object	Send to all relevant shards	Queries sent to shards
3	Search shards	Query on each shard	Each shard searches its data	Partial search results
4	Aggregate results	Partial search results	Combine and sort results	Final sorted search results
5	Return results	Final sorted search results	Send back to client	Search results displayed

💡 Search results returned to client, fulfilling the fast and scalable search need.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	Final
query	null	{"match":{"message":"search text"}}	Distributed to shards	Executed on shards	Partial results collected	Final sorted results

Key Moments - 2 Insights

Why can't traditional databases handle large search queries quickly?

Traditional databases are designed for structured data and transactions, not for full-text search at scale. As shown in execution_table step 1 and 2, Elasticsearch distributes queries to shards to handle large data efficiently.

How does Elasticsearch achieve fast search results?

Elasticsearch splits data into shards and searches them in parallel (step 3), then aggregates results quickly (step 4), enabling fast responses even with big data.

Visual Quiz - 3 Questions

Look at the execution_table, what happens at step 3?

A The query is parsed from JSON

B Results are aggregated and sorted

C Each shard searches its own data

D Results are returned to the client

💡 Hint

Check the 'Process' column at step 3 in execution_table.

According to variable_tracker, what is the state of 'query' after step 4?

A Partial results collected

B Executed on shards

C Distributed to shards

D Final sorted results

💡 Hint

Look at the 'query' row and the 'After Step 4' column in variable_tracker.

If Elasticsearch did not distribute queries to shards, which step would be missing?

A Step 1: Receive search request

B Step 2: Distribute query

C Step 4: Aggregate results

D Step 5: Return results

💡 Hint

Refer to execution_table step 2 description.

Concept Snapshot

Why Elasticsearch exists:
- Need: Fast search on large data
- Traditional DBs too slow for full-text search
- Elasticsearch: distributed, real-time search engine
- Splits data into shards for parallel search
- Aggregates results quickly for fast response

Full Transcript

Elasticsearch was created because people needed a way to search large amounts of data quickly. Traditional databases are not designed for fast full-text search on big data. Elasticsearch solves this by splitting data into smaller parts called shards and searching them at the same time. It then combines the results and sends them back fast. This makes searching large data sets easy and quick.

Practice

1. Why was Elasticsearch created?
Elasticsearch is mainly used because:

easy

A. It helps find and analyze large amounts of data quickly

B. It is a programming language for building websites

C. It stores data only in spreadsheets

D. It is used to design graphics and images

Solution

1. Step 1: Understand Elasticsearch's purpose

   Elasticsearch is designed to search and analyze data fast, especially large or complex data.

2. Step 2: Compare options with Elasticsearch's use

   The options claiming it is a programming language, stores data only in spreadsheets, or is used for graphics are incorrect.

3. Final Answer:

   It helps find and analyze large amounts of data quickly -> Option A

4. Quick Check:

   Elasticsearch = Fast data search and analysis [OK]

Hint: Elasticsearch = fast search and analysis tool [OK]

Common Mistakes:

• Thinking Elasticsearch is a programming language
• Confusing it with spreadsheet software
• Assuming it is for graphic design

2. Which of the following is the correct way to send a search query to Elasticsearch?

easy

A. Writing SQL queries directly in Elasticsearch console

B. Using JSON format over HTTP

C. Using XML files uploaded manually

D. Sending plain text emails to Elasticsearch server

Solution

1. Step 1: Identify Elasticsearch query method

   Elasticsearch uses JSON queries sent over HTTP to interact with data.

2. Step 2: Eliminate incorrect methods

   SQL, XML uploads, and emails are not how Elasticsearch receives queries.

3. Final Answer:

   Using JSON format over HTTP -> Option B

4. Quick Check:

   Elasticsearch queries = JSON + HTTP [OK]

Hint: Elasticsearch queries use JSON over HTTP [OK]

Common Mistakes:

• Assuming Elasticsearch uses SQL queries
• Thinking XML files are used for queries
• Believing email can send queries

3. What will Elasticsearch do when you send this JSON query?

{"query": {"match": {"title": "coffee"}}}

medium

A. Return an error because the query is invalid

B. Delete all documents with the title 'coffee'

C. Update the title field to 'coffee' in all documents

D. Search documents where the title contains the word 'coffee'

Solution

1. Step 1: Understand the JSON query structure

   The query uses "match" on the "title" field with the word "coffee" to find matching documents.

2. Step 2: Identify the query action

   This is a search query, not a delete or update command, so it returns matching documents.

3. Final Answer:

   Search documents where the title contains the word 'coffee' -> Option D

4. Quick Check:

   Match query = search for matching text [OK]

Hint: Match query finds documents containing the word [OK]

Common Mistakes:

• Confusing search with delete or update
• Thinking the query syntax is invalid
• Assuming it changes data instead of searching

4. You wrote this query to find documents with 'apple' in the description:

{"query": {"match": {"description": "apple"}}

But Elasticsearch returns an error. What is wrong?

medium

A. The field name 'description' is invalid

B. The word 'apple' is not allowed in queries

C. The JSON query is missing a closing brace '}'

D. Elasticsearch does not support match queries

Solution

1. Step 1: Check JSON syntax carefully

   The query is missing a closing brace '}' at the end, making it invalid JSON.

2. Step 2: Verify other options

   Words like 'apple' are allowed, 'description' can be a valid field, and match queries are supported.

3. Final Answer:

   The JSON query is missing a closing brace '}' -> Option C

4. Quick Check:

   JSON syntax error = missing brace [OK]

Hint: Always check JSON braces and commas carefully [OK]

Common Mistakes:

• Ignoring JSON syntax errors
• Assuming certain words are banned
• Thinking field names must be predefined

5. You have a large dataset with millions of records and want to quickly find all entries mentioning 'travel' in any text field. Why is Elasticsearch a good choice for this?

hard

A. Because it indexes data for fast full-text search across many fields

B. Because it stores data only in small files for quick access

C. Because it uses SQL joins to combine travel data efficiently

D. Because it compresses data to save disk space only

Solution

1. Step 1: Understand Elasticsearch's indexing

   Elasticsearch creates indexes that allow very fast full-text search across large datasets and multiple fields.

2. Step 2: Evaluate other options

   It does not rely on small files only, SQL joins, or just compression for speed.

3. Final Answer:

   Because it indexes data for fast full-text search across many fields -> Option A

4. Quick Check:

   Indexing = fast search on big data [OK]

Hint: Indexing enables fast search on large data [OK]

Common Mistakes:

• Thinking Elasticsearch uses SQL joins
• Believing it only compresses data
• Assuming it stores data in small files only