Inverted index data structure in Elasticsearch - Mini Project: Build & Apply

Building an Inverted Index in Elasticsearch

📖 Scenario: You are creating a simple search system for a small library. The library wants to quickly find books by keywords in their titles and descriptions. To do this, you will build an inverted index using Elasticsearch, which helps find documents containing specific words fast.

🎯 Goal: Build an inverted index in Elasticsearch by creating an index with documents representing books. Then configure the index to analyze text fields properly, add documents, and finally verify the inverted index structure.

📋 What You'll Learn

Create an Elasticsearch index named library with appropriate mappings for text fields.

Add a setting to use the standard analyzer for text fields.

Index three book documents with exact titles and descriptions provided.

Use the _termvectors API to retrieve the inverted index data for a specific document.

💡 Why This Matters

🌍 Real World

Inverted indexes are the backbone of search engines, enabling fast keyword searches in large text collections like libraries, websites, and databases.

💼 Career

Understanding how to build and query inverted indexes in Elasticsearch is essential for roles in search engineering, data engineering, and backend development.

1

Create the Elasticsearch index with mappings

Create an Elasticsearch index called library with mappings for two fields: title and description. Both fields should be of type text.

Elasticsearch

{
  "mappings": {
    "properties": {
      "title": { "type": "text" },
      "description": { "type": "text" }
    }
  }
}
# Your code here to create the index named 'library' with these mappings

Hint

Use the PUT method to create the index with the mappings section defining the fields.

2

Add analyzer settings to the index

Update the library index creation to include settings that specify the standard analyzer for both title and description fields.

Elasticsearch

PUT /library
{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "type": "standard"
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "title": { "type": "text" },
      "description": { "type": "text" }
    }
  }
}
# Your code here to add settings with standard analyzer

Hint

Include a settings section before mappings with the analyzer configuration.

3

Index three book documents

Add three documents to the library index with these exact data:
1. {"title": "Elasticsearch Basics", "description": "Learn the basics of Elasticsearch."}
2. {"title": "Advanced Elasticsearch", "description": "Deep dive into Elasticsearch features."}
3. {"title": "Search Engines", "description": "Overview of search engine technologies."}
Use document IDs 1, 2, and 3 respectively.

Elasticsearch

PUT /library/_doc/1
{
  "title": "Elasticsearch Basics",
  "description": "Learn the basics of Elasticsearch."
}

PUT /library/_doc/2
{
  "title": "Advanced Elasticsearch",
  "description": "Deep dive into Elasticsearch features."
}

PUT /library/_doc/3
{
  "title": "Search Engines",
  "description": "Overview of search engine technologies."
}
# Your code here to index these three documents

Hint

Use PUT /library/_doc/{id} to add each document with the exact fields and values.

4

Retrieve inverted index data for a document

Use the _termvectors API to get the inverted index terms for the document with ID 1 in the library index. Request term vectors for the title and description fields.

Elasticsearch

GET /library/_termvectors/1
{
  "fields": ["title", "description"]
}
# Your code here to retrieve term vectors for document 1

Hint

Use the GET /library/_termvectors/1 API with a JSON body specifying the fields.

Practice

1. What is the main purpose of an inverted index in Elasticsearch?

easy

A. To store documents in the order they were added

B. To map words to the documents where they appear

C. To compress data for storage efficiency

D. To encrypt documents for security

Solution

1. Step 1: Understand the role of inverted index

   An inverted index links each word to the documents containing it, enabling fast search.

2. Step 2: Compare options with definition

   Only To map words to the documents where they appear describes this mapping; others describe unrelated functions.

3. Final Answer:

   To map words to the documents where they appear -> Option B

4. Quick Check:

   Inverted index = word-to-document map [OK]

Hint: Remember: inverted index links words to documents [OK]

Common Mistakes:

• Thinking it stores documents in order
• Confusing with data compression
• Assuming it encrypts data

2. Which of the following is the correct way Elasticsearch uses an inverted index?

easy

A. It stores documents in a tree structure

B. It stores a list of words for each document

C. It stores documents as plain text files

D. It stores a list of documents for each unique word

Solution

1. Step 1: Recall inverted index structure

   Inverted index keeps a list of documents for each unique word found in the data.

2. Step 2: Match options to structure

   It stores a list of documents for each unique word matches this exactly; others describe different or incorrect storage methods.

3. Final Answer:

   It stores a list of documents for each unique word -> Option D

4. Quick Check:

   Inverted index = word to document list [OK]

Hint: Think: word first, then documents list [OK]

Common Mistakes:

• Confusing inverted index with forward index
• Assuming documents are stored as trees
• Thinking documents are stored as plain text only

3. Given these documents indexed in Elasticsearch:
Doc1: "apple banana apple"
Doc2: "banana orange"
What will the inverted index entry for the word banana contain?

medium

A. [Doc1, Doc2]

B. [Doc1]

C. [Doc2]

D. [] (empty list)

Solution

1. Step 1: Identify documents containing 'banana'

   Doc1 contains "banana" once, Doc2 contains "banana" once.

2. Step 2: List documents in inverted index for 'banana'

   Both Doc1 and Doc2 should be listed for the word "banana".

3. Final Answer:

   [Doc1, Doc2] -> Option A

4. Quick Check:

   banana appears in Doc1 and Doc2 [OK]

Hint: Check all docs containing the word, not just one [OK]

Common Mistakes:

• Listing only one document
• Confusing word frequency with document list
• Leaving the list empty

4. You have an inverted index that incorrectly shows the word orange linked only to Doc1, but Doc2 also contains "orange". What is the likely cause?

medium

A. The indexing process missed updating the inverted index for Doc2

B. The word "orange" does not exist in Doc2

C. Elasticsearch does not support multiple documents per word

D. The inverted index only stores the first document per word

Solution

1. Step 1: Verify word presence in documents

   Since "orange" is in Doc2, it should appear in the inverted index for that word.

2. Step 2: Identify indexing error

   If Doc2 is missing, likely the indexing process did not update the inverted index correctly for Doc2.

3. Final Answer:

   The indexing process missed updating the inverted index for Doc2 -> Option A

4. Quick Check:

   Missing doc means indexing error [OK]

Hint: Missing docs usually mean indexing skipped them [OK]

Common Mistakes:

• Assuming word is not in Doc2
• Thinking Elasticsearch limits docs per word
• Believing inverted index stores only one doc

5. You want to optimize search speed for a large text dataset in Elasticsearch. Which approach using the inverted index is best?

hard

A. Disable the inverted index to save storage space

B. Store all documents as plain text and scan each on search

C. Use the inverted index to quickly find documents containing search words

D. Index only the first word of each document

Solution

1. Step 1: Understand inverted index purpose

   It allows fast lookup of documents by words, speeding up search.

2. Step 2: Evaluate options for search speed

   Use the inverted index to quickly find documents containing search words uses the inverted index as intended; others slow search or lose data.

3. Final Answer:

   Use the inverted index to quickly find documents containing search words -> Option C

4. Quick Check:

   Inverted index speeds search by word lookup [OK]

Hint: Use inverted index for fast word-based search [OK]

Common Mistakes:

• Thinking scanning all text is faster
• Disabling inverted index to save space
• Indexing only part of documents