Token filters (lowercase, stemmer, synonym) in Elasticsearch - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this Elasticsearch analyzer test?

Given the following analyzer configuration and input text, what tokens will be produced after applying the lowercase and stemmer filters?

Elasticsearch

{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "standard",
          "filter": ["lowercase", "english_stemmer"]
        }
      },
      "filter": {
        "english_stemmer": {
          "type": "stemmer",
          "language": "english"
        }
      }
    }
  }
}

Input text: "Running runners run quickly"

A ["run", "run", "run", "quickli"]

B ["run", "runner", "run", "quickli"]

C ["running", "runners", "run", "quickly"]

D ["running", "runner", "run", "quick"]

💡 Hint

Remember that the lowercase filter converts all tokens to lowercase before stemming. The stemmer reduces words to their root form.

The lowercase filter converts all tokens to lowercase: 'Running' -> 'running', 'runners' -> 'runners', etc. The English stemmer then reduces 'running' and 'runners' to 'run', and 'quickly' to 'quickli'. So the final tokens are ['run', 'run', 'run', 'quickli'].

🧠 Conceptual

intermediate

Which token filter is responsible for replacing words with their synonyms?

In Elasticsearch, which token filter should you use to replace tokens with their synonyms during analysis?

A synonym

B stop

C stemmer

D lowercase

💡 Hint

Think about the filter that changes words to other words with similar meaning.

The synonym token filter replaces tokens with their synonyms based on a defined synonym list. Lowercase converts tokens to lowercase, stemmer reduces words to their root form, and stop removes common words.

🔧 Debug

advanced

Why does this synonym filter configuration cause an error?

Examine the following synonym filter configuration. Why does Elasticsearch reject it with a syntax error?

Elasticsearch

{
  "filter": {
    "my_synonym": {
      "type": "synonym",
      "synonyms": [
        "quick, fast",
        "jumps, leap"
      ]
    }
  }
}

A The synonyms list must be a file path, not an inline array.

B The filter type 'synonym' is deprecated and must be 'synonym_graph'.

C The synonym filter requires a 'synonyms_path' property instead of 'synonyms'.

D The synonyms must be separated by '=>' instead of commas.

💡 Hint

Check the latest Elasticsearch documentation for synonym filter types.

In recent Elasticsearch versions, the 'synonym' filter type is deprecated and replaced by 'synonym_graph' for better support in multi-term synonyms. Using 'synonym' causes a syntax error.

📝 Syntax

advanced

Which option correctly defines a lowercase token filter in Elasticsearch?

Select the correct JSON snippet that defines a lowercase token filter named 'my_lowercase'.

A { "filter": { "my_lowercase": { "type": "LowerCase" } } }

B { "filter": { "my_lowercase": { "type": "lower_case" } } }

C { "filters": { "my_lowercase": { "type": "lowercase" } } }

D { "filter": { "my_lowercase": { "type": "lowercase" } } }

💡 Hint

Check the exact spelling and case sensitivity of the 'type' property.

The correct key is 'filter' (not 'filters'), and the type must be exactly 'lowercase' in lowercase letters without underscores. Option D matches this exactly.

🚀 Application

expert

How many tokens are produced by this analyzer with synonym and lowercase filters?

Given this analyzer configuration and input, how many tokens will be produced?

Elasticsearch

{
  "settings": {
    "analysis": {
      "analyzer": {
        "syn_lower_analyzer": {
          "tokenizer": "standard",
          "filter": ["lowercase", "my_synonym"]
        }
      },
      "filter": {
        "my_synonym": {
          "type": "synonym_graph",
          "synonyms": [
            "quick, fast",
            "jumps, leaps"
          ]
        }
      }
    }
  }
}

Input text: "The quick fox jumps"

A 4

B 5

C 6

D 7

💡 Hint

Remember that synonym_graph can produce multiple tokens for synonyms, increasing token count.

The input tokens after standard tokenizer and lowercase are: ['the', 'quick', 'fox', 'jumps']. The synonym_graph filter expands 'quick' to ['quick', 'fast'] and 'jumps' to ['jumps', 'leaps'], so tokens become: ['the', 'quick', 'fast', 'fox', 'jumps', 'leaps'], total 6 tokens.

Practice

1.

What does the lowercase token filter do in Elasticsearch?

easy

A. Converts all tokens to lowercase letters

B. Removes stop words from the tokens

C. Finds the root form of tokens

D. Adds synonyms to tokens

Solution

1. Step 1: Understand the lowercase filter purpose

   The lowercase filter changes all tokens to lowercase to make searches case-insensitive.

2. Step 2: Compare with other filters

   Stemmer finds root words, synonym adds similar words, stop word filter removes common words. Only lowercase converts to lowercase.

3. Final Answer:

   Converts all tokens to lowercase letters -> Option A

4. Quick Check:

   Lowercase filter = converts tokens to lowercase [OK]

Hint: Lowercase filter always makes words small letters [OK]

Common Mistakes:

• Confusing lowercase with stemmer
• Thinking lowercase adds synonyms
• Mixing lowercase with stop word removal

2.

Which of the following is the correct syntax to define a synonym token filter in Elasticsearch?

{
  "filter": {
    "my_synonym": {
      "type": "synonym",
      "synonyms": ["quick,fast"]
    }
  }
}

easy

A. { "type": "stemmer", "synonyms": ["quick,fast"] }

B. { "type": "stop", "synonyms": ["quick,fast"] }

C. { "type": "synonym", "synonyms": ["quick,fast"] }

D. { "type": "lowercase", "synonyms": ["quick,fast"] }

Solution

1. Step 1: Identify the token filter type for synonyms

   The synonym filter must have "type": "synonym" to work correctly.

2. Step 2: Check the synonyms property

   The synonyms list is correctly provided as an array of strings with comma-separated words.

3. Final Answer:

   { "type": "synonym", "synonyms": ["quick,fast"] } -> Option C

4. Quick Check:

   Synonym filter syntax = type synonym with synonyms list [OK]

Hint: Synonym filter always uses type 'synonym' with synonyms array [OK]

Common Mistakes:

• Using wrong type like stemmer or lowercase
• Not providing synonyms as an array
• Misspelling 'synonym' in type

3.

Given this analyzer configuration in Elasticsearch:

{
  "analyzer": {
    "my_analyzer": {
      "tokenizer": "standard",
      "filter": ["lowercase", "porter_stem"]
    }
  }
}

What will be the output tokens for the input text "Running Runs"?

medium

A. ["run", "runs"]

B. ["running", "runs"]

C. ["Running", "Runs"]

D. ["run", "run"]

Solution

1. Step 1: Apply lowercase filter

   Input "Running Runs" becomes "running runs" after lowercase filter.

2. Step 2: Apply porter_stem filter

   Porter stemmer reduces "running" and "runs" to root "run" for both tokens.

3. Final Answer:

   ["run", "run"] -> Option D

4. Quick Check:

   Lowercase + stemmer = root forms in lowercase [OK]

Hint: Lowercase then stemmer gives root lowercase tokens [OK]

Common Mistakes:

• Ignoring stemming effect
• Not applying lowercase first
• Expecting original case tokens

4.

Identify the error in this token filter definition:

{
  "filter": {
    "my_stemmer": {
      "type": "stemmer",
      "language": "englishy"
    }
  }
}

medium

A. "type" should be "stemmer_override"

B. "language" value is misspelled; should be "english"

C. "filter" key must be inside "analyzer"

D. Missing "synonyms" array

Solution

1. Step 1: Check the "language" parameter spelling

   The language "englishy" is incorrect; the correct value is "english".

2. Step 2: Verify the stemmer type and parameters

   Type "stemmer" is correct; no synonyms needed here; filter can be defined outside analyzer.

3. Final Answer:

   "language" value is misspelled; should be "english" -> Option B

4. Quick Check:

   Stemmer language spelling must be exact [OK]

Hint: Check spelling of language names in stemmer filters [OK]

Common Mistakes:

• Using invalid language names
• Confusing stemmer with synonym filter
• Expecting synonyms in stemmer filter

5.

You want to create an analyzer that makes searches case-insensitive, finds root forms of words, and also matches synonyms like "fast" for "quick". Which token filters should you include in your analyzer?

hard

A. ["lowercase", "stemmer", "synonym"]

B. ["stop", "lowercase", "synonym"]

C. ["lowercase", "synonym"]

D. ["stemmer", "stop"]

Solution

1. Step 1: Identify filters for case-insensitivity and root forms

   Lowercase filter makes tokens lowercase; stemmer finds root forms.

2. Step 2: Add synonym filter for matching similar words

   Synonym filter adds synonyms like "fast" for "quick" to improve matching.

3. Final Answer:

   ["lowercase", "stemmer", "synonym"] -> Option A

4. Quick Check:

   Lowercase + stemmer + synonym = case-insensitive, root, and synonym matching [OK]

Hint: Combine lowercase, stemmer, and synonym filters for full matching [OK]

Common Mistakes:

• Omitting stemmer or synonym filters
• Using stop filter instead of stemmer
• Not including lowercase for case-insensitivity