Why Autocomplete with edge n-gram in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if your search box could guess your words instantly, saving you time and frustration?

The Scenario

Imagine typing a search query on a website and waiting for the page to reload every time you add a letter, hoping it guesses what you want.

Without autocomplete, you must type the full word and submit the search to see results.

The Problem

This manual approach is slow and frustrating because the system can only search after you finish typing.

It also makes many mistakes because it can't suggest partial matches as you type.

Users often get annoyed and leave the site.

The Solution

Autocomplete with edge n-gram breaks words into smaller pieces starting from the beginning, so the system can quickly suggest matches as you type each letter.

This makes searching fast and smooth, giving instant suggestions and improving user experience.

Before vs After

✗ Before

GET /products/_search
{
  "query": {
    "match": {
      "name": "lap"
    }
  }
}

✓ After

PUT /products
{
  "settings": {
    "analysis": {
      "analyzer": {
        "autocomplete": {
          "tokenizer": "autocomplete_tokenizer"
        }
      },
      "tokenizer": {
        "autocomplete_tokenizer": {
          "type": "edge_ngram",
          "min_gram": 1,
          "max_gram": 20,
          "token_chars": ["letter"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "name": {
        "type": "text",
        "analyzer": "autocomplete",
        "search_analyzer": "standard"
      }
    }
  }
}

What It Enables

It enables instant, smart search suggestions that update as you type, making search faster and more user-friendly.

Real Life Example

When you start typing "lap" in an online store's search box, autocomplete with edge n-gram suggests "laptop", "lapel pin", or "lap desk" immediately, helping you find products faster.

Key Takeaways

Manual search waits for full input, causing delays and frustration.

Edge n-gram breaks words into starting pieces for quick matching.

Autocomplete with edge n-gram improves speed and user experience by suggesting results as you type.

Practice

1. What is the main purpose of using an edge_ngram tokenizer in Elasticsearch autocomplete?

easy

A. To break words into prefixes for fast autocomplete suggestions

B. To sort search results alphabetically

C. To remove stop words from the search query

D. To index only complete words without partial matches

Solution

1. Step 1: Understand edge n-gram tokenizer function

   The edge n-gram tokenizer splits words into prefixes, like 'app', 'appl', 'apple' for 'apple'.

2. Step 2: Relate tokenizer to autocomplete

   This prefix splitting allows Elasticsearch to quickly suggest words as users type partial input.

3. Final Answer:

   To break words into prefixes for fast autocomplete suggestions -> Option A

4. Quick Check:

   edge_ngram tokenizer = prefix splitting [OK]

Hint: Edge n-gram creates prefixes for autocomplete speed [OK]

Common Mistakes:

• Thinking edge n-gram sorts results
• Confusing with stop word removal
• Assuming it indexes only full words

2. Which of the following is the correct way to define an edge_ngram tokenizer in Elasticsearch settings?

easy

A. {
"tokenizer": {
"my_edge_ngram": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 5,
"token_chars": ["letter"]
}
}
}

B. {
"tokenizer": {
"my_edge_ngram": {
"type": "standard",
"min_gram": 2,
"max_gram": 5
}
}
}

C. {
"analyzer": {
"my_edge_ngram": {
"type": "edge_ngram"
}
}
}

D. {
"tokenizer": {
"my_edge_ngram": {
"type": "ngram",
"min_gram": 1,
"max_gram": 3
}
}
}

Solution

1. Step 1: Identify correct tokenizer type and parameters

   The edge_ngram tokenizer requires type 'edge_ngram' with min_gram and max_gram settings.

2. Step 2: Check token_chars usage

   { "tokenizer": { "my_edge_ngram": { "type": "edge_ngram", "min_gram": 2, "max_gram": 5, "token_chars": ["letter"] } } } correctly sets token_chars to 'letter' to include only letters in tokens.

3. Final Answer:

   Option A with type 'edge_ngram' and proper min_gram, max_gram -> Option A

4. Quick Check:

   edge_ngram tokenizer syntax = { "tokenizer": { "my_edge_ngram": { "type": "edge_ngram", "min_gram": 2, "max_gram": 5, "token_chars": ["letter"] } } } [OK]

Hint: Look for type 'edge_ngram' with min_gram and max_gram [OK]

Common Mistakes:

• Using 'standard' type instead of 'edge_ngram'
• Defining edge_ngram under analyzer instead of tokenizer
• Confusing ngram with edge_ngram tokenizer

3. Given this analyzer configuration using edge n-gram tokenizer: { "settings": { "analysis": { "tokenizer": { "autocomplete_tokenizer": { "type": "edge_ngram", "min_gram": 1, "max_gram": 3, "token_chars": ["letter"] } }, "analyzer": { "autocomplete": { "type": "custom", "tokenizer": "autocomplete_tokenizer", "filter": ["lowercase"] } } } }} What tokens will be generated for the word "cat" during indexing?

medium

A. ["cat"]

B. ["c", "a", "t"]

C. ["ca", "cat", "cat"]

D. ["c", "ca", "cat"]

Solution

1. Step 1: Understand edge n-gram tokenizer with min_gram=1 and max_gram=3

   It creates tokens from the start of the word with lengths 1 to 3 letters.

2. Step 2: Generate tokens for 'cat'

   Tokens are 'c' (1 letter), 'ca' (2 letters), and 'cat' (3 letters).

3. Final Answer:

   ["c", "ca", "cat"] -> Option D

4. Quick Check:

   edge n-gram tokens for 'cat' = ["c", "ca", "cat"] [OK]

Hint: Edge n-gram tokens start from first letter up to max_gram [OK]

Common Mistakes:

• Assuming only full word token is generated
• Including tokens from middle or end of word
• Confusing n-gram with edge n-gram tokens

4. You have this analyzer configuration for autocomplete but your search results do not return expected matches: { "settings": { "analysis": { "tokenizer": { "autocomplete_tokenizer": { "type": "edge_ngram", "min_gram": 2, "max_gram": 5 } }, "analyzer": { "autocomplete": { "type": "custom", "tokenizer": "autocomplete_tokenizer", "filter": ["lowercase"] }, "search_analyzer": { "type": "standard" } } } }} What is the likely cause of the problem?

medium

A. The tokenizer type should be 'ngram' not 'edge_ngram'

B. The search analyzer is incorrectly defined as 'standard' instead of using the same analyzer

C. The min_gram value is too low causing too many tokens

D. The filter 'lowercase' should not be used in the analyzer

Solution

1. Step 1: Understand analyzer and search_analyzer roles

   Indexing uses edge n-gram tokenizer, but searching uses standard analyzer which does not generate prefixes.

2. Step 2: Identify mismatch causing no autocomplete matches

   Because search analyzer does not tokenize input into prefixes, queries won't match edge n-gram tokens.

3. Final Answer:

   Search analyzer should match or complement edge n-gram analyzer, not standard -> Option B

4. Quick Check:

   Analyzer mismatch = The search analyzer is incorrectly defined as 'standard' instead of using the same analyzer [OK]

Hint: Search analyzer must complement edge n-gram tokenizer [OK]

Common Mistakes:

• Changing min_gram instead of fixing analyzer mismatch
• Confusing ngram and edge_ngram tokenizer types
• Removing lowercase filter unnecessarily

5. You want to implement autocomplete for a product search using edge n-gram tokenizer. Which approach correctly combines indexing and searching analyzers for best autocomplete results?

hard

A. Use edge n-gram tokenizer in the indexing analyzer and standard analyzer in the search analyzer

B. Use edge n-gram tokenizer in both indexing and search analyzers

C. Use edge n-gram tokenizer in indexing analyzer and a lowercase + standard tokenizer in search analyzer

D. Use standard tokenizer in indexing analyzer and edge n-gram tokenizer in search analyzer

Solution

1. Step 1: Understand indexing with edge n-gram tokenizer

   Indexing uses edge n-gram tokenizer to create prefix tokens for autocomplete.

2. Step 2: Understand searching with standard tokenizer plus lowercase filter

   Search analyzer uses standard tokenizer to tokenize user input normally and lowercase filter to match indexed tokens.

3. Step 3: Why not use edge n-gram tokenizer in search analyzer?

   Using edge n-gram in search analyzer can cause unexpected partial matches and performance issues.

4. Final Answer:

   Index with edge n-gram, search with standard + lowercase analyzer -> Option C

5. Quick Check:

   Index edge n-gram + search standard = Use edge n-gram tokenizer in indexing analyzer and a lowercase + standard tokenizer in search analyzer [OK]

Hint: Index with edge n-gram, search with standard + lowercase [OK]

Common Mistakes:

• Using edge n-gram tokenizer in search analyzer
• Using standard tokenizer in indexing analyzer
• Not applying lowercase filter in search analyzer