Synonym handling in Elasticsearch - Step-by-Step Execution

Concept Flow - Synonym handling

User Query Input

↓

Analyze Query Text

↓

Apply Synonym Filter

↓

Expand Query with Synonyms

↓

Search Expanded Query in Index

↓

Return Matching Documents

The query text is analyzed and synonyms are applied to expand the search terms, improving search results by including related words.

Execution Sample

Elasticsearch

PUT /my_index
{
  "settings": {
    "analysis": {
      "filter": {
        "my_synonym_filter": {
          "type": "synonym",
          "synonyms": ["quick,fast"]
        }
      },
      "analyzer": {
        "my_synonym_analyzer": {
          "tokenizer": "standard",
          "filter": ["lowercase", "my_synonym_filter"]
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "analyzer": "my_synonym_analyzer"
      }
    }
  }
}

GET /my_index/_search
{
  "query": {
    "match": {
      "content": "fast car"
    }
  }
}

Defines an index with a synonym filter that treats 'quick' and 'fast' as the same, then searches for 'fast car' which also matches documents with 'quick car'.

Execution Table

Step	Input Text	Analyzer Action	Tokens After Analysis	Query Expansion
1	fast car	Tokenize and lowercase	["fast", "car"]	Apply synonym filter
2	fast	Synonym filter expands	["fast", "quick"]	Expand query to include 'quick'
3	car	No synonym	["car"]	No expansion
4	Final tokens	Combine tokens	["fast", "quick", "car"]	Search with expanded terms
5	Search	Match documents with any token	Documents with 'fast car' or 'quick car'	Return results

💡 All tokens processed and query expanded with synonyms for search

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
tokens	[]	["fast", "car"]	["fast", "quick", "car"]	["fast", "quick", "car"]	["fast", "quick", "car"]
query_terms	[]	["fast", "car"]	["fast", "quick", "car"]	["fast", "quick", "car"]	["fast", "quick", "car"]

Key Moments - 3 Insights

Why does the token 'fast' become two tokens 'fast' and 'quick' after synonym filter?

Because the synonym filter expands 'fast' to include its synonym 'quick', so the query searches for both words to find more matches (see execution_table step 2).

Does the synonym filter change the original tokens or add new ones?

It adds new tokens alongside the original ones, so both 'fast' and 'quick' are searched, not replacing 'fast' (see execution_table step 2 and 4).

Why is the analyzer applied at both index and search time?

To ensure that synonyms are recognized consistently when indexing documents and when searching, so queries match documents correctly.

Visual Quiz - 3 Questions

Look at the execution_table at step 2, what tokens are present after applying the synonym filter to 'fast'?

A ["fast"]

B ["fast", "quick"]

C ["quick"]

D ["fast", "car"]

💡 Hint

Check the 'Tokens After Analysis' column at step 2 in the execution_table.

At which step does the query get expanded to include synonyms?

A Step 2

B Step 1

C Step 3

D Step 5

💡 Hint

Look for when the synonym filter is applied and tokens increase in the execution_table.

If the synonym filter was removed, how would the tokens after step 2 change?

A ["fast", "quick", "car"]

B ["quick", "car"]

C ["fast", "car"]

D ["car"]

💡 Hint

Without synonym filter, tokens remain as originally tokenized (see variable_tracker start and after step 1).

Concept Snapshot

Synonym handling in Elasticsearch:
- Define a synonym filter in analysis settings
- Use it in a custom analyzer
- Analyzer expands tokens with synonyms at index and search time
- Queries match documents with original or synonym terms
- Improves search recall by including related words

Full Transcript

Synonym handling in Elasticsearch means when you search for a word, the system also looks for its synonyms to find more results. The process starts when the user inputs a query. The query text is analyzed by breaking it into tokens and converting to lowercase. Then the synonym filter is applied, which expands tokens like 'fast' to include 'quick'. This expansion means the search will look for documents containing either word. The final expanded tokens are used to search the index, returning documents that match any of the tokens. This helps find more relevant documents even if they use different words with the same meaning. The synonym filter adds new tokens alongside original ones, not replacing them. It is important to apply the same analyzer at index and search time to keep consistency. This way, synonyms improve search results by broadening the matching terms.

Practice

1. What is the main purpose of synonym handling in Elasticsearch?

easy

A. To find search results that include different words with the same meaning

B. To speed up the indexing process

C. To store data in multiple languages automatically

D. To encrypt search queries for security

Solution

1. Step 1: Understand synonym handling concept

   Synonym handling allows Elasticsearch to match words that mean the same, improving search results.

2. Step 2: Identify the main goal

   The goal is to find results with different words but same meaning, not speed or encryption.

3. Final Answer:

   To find search results that include different words with the same meaning -> Option A

4. Quick Check:

   Synonym handling = find similar meaning words [OK]

Hint: Synonyms help find words with same meaning [OK]

Common Mistakes:

• Thinking synonyms speed up indexing
• Confusing synonyms with encryption
• Assuming synonyms translate languages automatically

2. Which of the following is the correct way to define a synonym filter in Elasticsearch settings?

easy

A. "filter": { "synonym_filter": { "type": "keyword", "synonyms": ["quick,fast"] } }

B. "filter": { "synonym_filter": { "type": "stop", "synonyms": ["quick,fast"] } }

C. "filter": { "synonym_filter": { "type": "synonym", "synonyms": ["quick,fast"] } }

D. "filter": { "synonym_filter": { "type": "stemmer", "synonyms": ["quick,fast"] } }

Solution

1. Step 1: Identify correct filter type for synonyms

   The filter type for synonyms must be "synonym" to handle synonym lists.

2. Step 2: Check the filter definition syntax

   "filter": { "synonym_filter": { "type": "synonym", "synonyms": ["quick,fast"] } } correctly uses "type": "synonym" and defines synonyms as a list.

3. Final Answer:

   "filter": { "synonym_filter": { "type": "synonym", "synonyms": ["quick,fast"] } } -> Option C

4. Quick Check:

   Synonym filter type = "synonym" [OK]

Hint: Synonym filters use type "synonym" [OK]

Common Mistakes:

• Using wrong filter types like stop or stemmer
• Not defining synonyms as a list
• Misspelling "synonym" type

3. Given this analyzer configuration, what will be the tokens produced for the text "fast car"?

{
  "analysis": {
    "filter": {
      "synonym_filter": {
        "type": "synonym",
        "synonyms": ["fast,quick"]
      }
    },
    "analyzer": {
      "synonym_analyzer": {
        "tokenizer": "standard",
        "filter": ["lowercase", "synonym_filter"]
      }
    }
  }
}

medium

A. ["fast", "quick", "car", "car"]

B. ["fast", "quick", "car"]

C. ["quick", "car"]

D. ["fast", "car"]

Solution

1. Step 1: Analyze the text with the synonym filter

   The synonym filter maps "fast" to "quick", so both tokens appear for "fast".

2. Step 2: List all tokens after analysis

   Tokens are "fast", "quick" (synonym), and "car" (unchanged).

3. Final Answer:

   ["fast", "quick", "car"] -> Option B

4. Quick Check:

   Synonym adds "quick" token for "fast" [OK]

Hint: Synonym filter adds synonyms as extra tokens [OK]

Common Mistakes:

• Ignoring synonym addition and listing only original tokens
• Replacing original token instead of adding synonym
• Duplicating tokens unnecessarily

4. You defined a synonym filter but your search results do not include synonyms. What is the most likely mistake?

medium

A. You used the synonym filter in the index but not in the search analyzer

B. You forgot to add the synonym filter to the analyzer's filter list

C. You defined synonyms as a string instead of a list

D. You used the synonym filter with the wrong tokenizer

Solution

1. Step 1: Understand how synonym filters affect search

   Synonyms must be applied in the analyzer used during search to affect results.

2. Step 2: Identify common setup mistake

   Often synonyms are applied only at index time, missing search analyzer usage.

3. Final Answer:

   You used the synonym filter in the index but not in the search analyzer -> Option A

4. Quick Check:

   Synonyms must be in search analyzer too [OK]

Hint: Apply synonym filter in both index and search analyzers [OK]

Common Mistakes:

• Adding synonym filter only at index time
• Not including synonym filter in search analyzer
• Misconfiguring synonyms as string instead of list

5. You want to improve search for a product catalog where users may search for "tv", "television", or "smart tv". How should you configure synonyms to handle these terms effectively?

hard

A. Apply synonyms only at search time to avoid changing indexed data

B. Define synonyms only for "tv" and "television" but exclude "smart tv" because it has two words

C. Use a stop filter to remove "smart" and then define synonyms for "tv" and "television"

D. Define synonyms as: "tv, television, smart tv" in a synonym filter and apply it in both index and search analyzers

Solution

1. Step 1: Define all related terms as synonyms

   Include "tv", "television", and "smart tv" together to cover all user queries.

2. Step 2: Apply synonym filter in both index and search analyzers

   This ensures synonyms work during indexing and searching for best results.

3. Final Answer:

   Define synonyms as: "tv, television, smart tv" in a synonym filter and apply it in both index and search analyzers -> Option D

4. Quick Check:

   Synonyms cover all terms and apply at both stages [OK]

Hint: Include all terms and apply synonyms at index and search [OK]

Common Mistakes:

• Ignoring multi-word synonyms like "smart tv"
• Applying synonyms only at search time
• Using stop filter incorrectly to remove words