Inverted index data structure in Elasticsearch - Time & Space Complexity

Time Complexity: Inverted index data structure

O(n)

Understanding Time Complexity

When searching text in Elasticsearch, the inverted index helps find words quickly.

We want to know how the search time changes as the amount of text grows.

Scenario Under Consideration

Analyze the time complexity of building and searching an inverted index.

{
  "mappings": {
    "properties": {
      "content": { "type": "text" }
    }
  }
}

GET /my_index/_search
{
  "query": {
    "match": { "content": "search term" }
  }
}
    

This code defines a text field that Elasticsearch indexes using an inverted index, then searches for documents containing "search term".

Identify Repeating Operations

Look at what repeats when building and searching the inverted index.

• Primary operation: Scanning each word in all documents to build the index.
• How many times: Once per word per document when indexing; then searching looks up words in the index.

How Execution Grows With Input

As you add more documents or words, indexing takes longer because it reads all words.

Input Size (n words)	Approx. Operations
10	About 10 word scans
100	About 100 word scans
1000	About 1000 word scans

Pattern observation: The work grows directly with the number of words; doubling words doubles work.

Final Time Complexity

Time Complexity: O(n)

This means the time to build the index grows in a straight line with the number of words.

Common Mistake

[X] Wrong: "Searching the inverted index takes as long as reading all documents again."

[OK] Correct: The inverted index lets Elasticsearch jump directly to matching words, so search is much faster than scanning all documents.

Interview Connect

Understanding how inverted indexes scale helps you explain efficient search systems clearly and confidently.

Self-Check

What if we added a second word to search for? How would the time complexity change?

Practice

1. What is the main purpose of an inverted index in Elasticsearch?

easy

A. To store documents in the order they were added

B. To map words to the documents where they appear

C. To compress data for storage efficiency

D. To encrypt documents for security

Solution

1. Step 1: Understand the role of inverted index

   An inverted index links each word to the documents containing it, enabling fast search.

2. Step 2: Compare options with definition

   Only To map words to the documents where they appear describes this mapping; others describe unrelated functions.

3. Final Answer:

   To map words to the documents where they appear -> Option B

4. Quick Check:

   Inverted index = word-to-document map [OK]

Hint: Remember: inverted index links words to documents [OK]

Common Mistakes:

• Thinking it stores documents in order
• Confusing with data compression
• Assuming it encrypts data

2. Which of the following is the correct way Elasticsearch uses an inverted index?

easy

A. It stores documents in a tree structure

B. It stores a list of words for each document

C. It stores documents as plain text files

D. It stores a list of documents for each unique word

Solution

1. Step 1: Recall inverted index structure

   Inverted index keeps a list of documents for each unique word found in the data.

2. Step 2: Match options to structure

   It stores a list of documents for each unique word matches this exactly; others describe different or incorrect storage methods.

3. Final Answer:

   It stores a list of documents for each unique word -> Option D

4. Quick Check:

   Inverted index = word to document list [OK]

Hint: Think: word first, then documents list [OK]

Common Mistakes:

• Confusing inverted index with forward index
• Assuming documents are stored as trees
• Thinking documents are stored as plain text only

3. Given these documents indexed in Elasticsearch:
Doc1: "apple banana apple"
Doc2: "banana orange"
What will the inverted index entry for the word banana contain?

medium

A. [Doc1, Doc2]

B. [Doc1]

C. [Doc2]

D. [] (empty list)

Solution

1. Step 1: Identify documents containing 'banana'

   Doc1 contains "banana" once, Doc2 contains "banana" once.

2. Step 2: List documents in inverted index for 'banana'

   Both Doc1 and Doc2 should be listed for the word "banana".

3. Final Answer:

   [Doc1, Doc2] -> Option A

4. Quick Check:

   banana appears in Doc1 and Doc2 [OK]

Hint: Check all docs containing the word, not just one [OK]

Common Mistakes:

• Listing only one document
• Confusing word frequency with document list
• Leaving the list empty

4. You have an inverted index that incorrectly shows the word orange linked only to Doc1, but Doc2 also contains "orange". What is the likely cause?

medium

A. The indexing process missed updating the inverted index for Doc2

B. The word "orange" does not exist in Doc2

C. Elasticsearch does not support multiple documents per word

D. The inverted index only stores the first document per word

Solution

1. Step 1: Verify word presence in documents

   Since "orange" is in Doc2, it should appear in the inverted index for that word.

2. Step 2: Identify indexing error

   If Doc2 is missing, likely the indexing process did not update the inverted index correctly for Doc2.

3. Final Answer:

   The indexing process missed updating the inverted index for Doc2 -> Option A

4. Quick Check:

   Missing doc means indexing error [OK]

Hint: Missing docs usually mean indexing skipped them [OK]

Common Mistakes:

• Assuming word is not in Doc2
• Thinking Elasticsearch limits docs per word
• Believing inverted index stores only one doc

5. You want to optimize search speed for a large text dataset in Elasticsearch. Which approach using the inverted index is best?

hard

A. Disable the inverted index to save storage space

B. Store all documents as plain text and scan each on search

C. Use the inverted index to quickly find documents containing search words

D. Index only the first word of each document

Solution

1. Step 1: Understand inverted index purpose

   It allows fast lookup of documents by words, speeding up search.

2. Step 2: Evaluate options for search speed

   Use the inverted index to quickly find documents containing search words uses the inverted index as intended; others slow search or lose data.

3. Final Answer:

   Use the inverted index to quickly find documents containing search words -> Option C

4. Quick Check:

   Inverted index speeds search by word lookup [OK]

Hint: Use inverted index for fast word-based search [OK]

Common Mistakes:

• Thinking scanning all text is faster
• Disabling inverted index to save space
• Indexing only part of documents