Token filters (lowercase, stemmer, synonym) in Elasticsearch - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to add a lowercase token filter in the analyzer.

Elasticsearch

{
  "settings": {
    "analysis": {
      "analyzer": {
        "my_analyzer": {
          "tokenizer": "standard",
          "filter": ["[1]"]
        }
      },
      "filter": {
        "lowercase_filter": {
          "type": "lowercase"
        }
      }
    }
  }
}

Drag options to blanks, or click blank then click option'

Asynonym_filter

Bstemmer_filter

Clowercase_filter

Dstop_filter

Attempts:

3 left

💡 Hint

Common Mistakes

Using a filter name that is not defined in the filter section.

Confusing filter names like stemmer or synonym instead of lowercase.

The lowercase token filter converts all tokens to lowercase. Here, the filter named lowercase_filter is added to the analyzer's filter list.

2fill in blank

medium

Complete the code to add a stemmer token filter named "english_stemmer".

Elasticsearch

{
  "settings": {
    "analysis": {
      "filter": {
        "english_stemmer": {
          "type": "[1]",
          "language": "english"
        }
      }
    }
  }
}

Drag options to blanks, or click blank then click option'

Astemmer

Blowercase

Cstop

Dsynonym

Attempts:

3 left

💡 Hint

Common Mistakes

Using "stemmer_filter" instead of "stemmer" as the type.

Confusing stemmer with synonym or lowercase types.

The stemmer filter reduces words to their root form. The type for this filter is "stemmer".

3fill in blank

hard

Fix the error in the synonym filter definition by completing the missing type.

Elasticsearch

{
  "settings": {
    "analysis": {
      "filter": {
        "my_synonym_filter": {
          "type": "[1]",
          "synonyms": [
            "quick,fast",
            "jumps,leaps"
          ]
        }
      }
    }
  }
}

Drag options to blanks, or click blank then click option'

Asynonym

Bstemmer

Clowercase

Dstop

Attempts:

3 left

💡 Hint

Common Mistakes

Using "synonym_filter" instead of "synonym" as the type.

Confusing synonym filter type with stemmer or lowercase.

The synonym filter type must be set to "synonym" to work correctly with synonym lists.

4fill in blank

hard

Fill both blanks to create an analyzer that uses the standard tokenizer and applies lowercase and stemmer filters.

Elasticsearch

{
  "settings": {
    "analysis": {
      "analyzer": {
        "custom_analyzer": {
          "tokenizer": "[1]",
          "filter": ["lowercase_filter", "[2]"]
        }
      },
      "filter": {
        "lowercase_filter": {
          "type": "lowercase"
        },
        "english_stemmer": {
          "type": "stemmer",
          "language": "english"
        }
      }
    }
  }
}

Drag options to blanks, or click blank then click option'

Astandard

Benglish_stemmer

Ckeyword

Dsynonym_filter

Attempts:

3 left

💡 Hint

Common Mistakes

Using "keyword" tokenizer instead of "standard".

Using filter names not defined in the filter section.

The standard tokenizer splits text into words. The filters applied are lowercase and the English stemmer filter.

5fill in blank

hard

Fill all three blanks to define a synonym filter and use it in an analyzer with lowercase and standard tokenizer.

Elasticsearch

{
  "settings": {
    "analysis": {
      "filter": {
        "my_synonym_filter": {
          "type": "[1]",
          "synonyms": ["fast,quick", "jumps,leaps"]
        }
      },
      "analyzer": {
        "synonym_analyzer": {
          "tokenizer": "[2]",
          "filter": ["[3]", "my_synonym_filter"]
        }
      }
    }
  }
}

Drag options to blanks, or click blank then click option'

Asynonym

Bstandard

Clowercase

Dstemmer

Attempts:

3 left

💡 Hint

Common Mistakes

Using incorrect filter types like "stemmer" for synonyms.

Placing filters in wrong order or using undefined filters.

The synonym filter type is "synonym". The analyzer uses the "standard" tokenizer and applies the "lowercase" filter before the synonym filter.

Practice

1.

What does the lowercase token filter do in Elasticsearch?

easy

A. Converts all tokens to lowercase letters

B. Removes stop words from the tokens

C. Finds the root form of tokens

D. Adds synonyms to tokens

Solution

1. Step 1: Understand the lowercase filter purpose

   The lowercase filter changes all tokens to lowercase to make searches case-insensitive.

2. Step 2: Compare with other filters

   Stemmer finds root words, synonym adds similar words, stop word filter removes common words. Only lowercase converts to lowercase.

3. Final Answer:

   Converts all tokens to lowercase letters -> Option A

4. Quick Check:

   Lowercase filter = converts tokens to lowercase [OK]

Hint: Lowercase filter always makes words small letters [OK]

Common Mistakes:

• Confusing lowercase with stemmer
• Thinking lowercase adds synonyms
• Mixing lowercase with stop word removal

2.

Which of the following is the correct syntax to define a synonym token filter in Elasticsearch?

{
  "filter": {
    "my_synonym": {
      "type": "synonym",
      "synonyms": ["quick,fast"]
    }
  }
}

easy

A. { "type": "stemmer", "synonyms": ["quick,fast"] }

B. { "type": "stop", "synonyms": ["quick,fast"] }

C. { "type": "synonym", "synonyms": ["quick,fast"] }

D. { "type": "lowercase", "synonyms": ["quick,fast"] }

Solution

1. Step 1: Identify the token filter type for synonyms

   The synonym filter must have "type": "synonym" to work correctly.

2. Step 2: Check the synonyms property

   The synonyms list is correctly provided as an array of strings with comma-separated words.

3. Final Answer:

   { "type": "synonym", "synonyms": ["quick,fast"] } -> Option C

4. Quick Check:

   Synonym filter syntax = type synonym with synonyms list [OK]

Hint: Synonym filter always uses type 'synonym' with synonyms array [OK]

Common Mistakes:

• Using wrong type like stemmer or lowercase
• Not providing synonyms as an array
• Misspelling 'synonym' in type

3.

Given this analyzer configuration in Elasticsearch:

{
  "analyzer": {
    "my_analyzer": {
      "tokenizer": "standard",
      "filter": ["lowercase", "porter_stem"]
    }
  }
}

What will be the output tokens for the input text "Running Runs"?

medium

A. ["run", "runs"]

B. ["running", "runs"]

C. ["Running", "Runs"]

D. ["run", "run"]

Solution

1. Step 1: Apply lowercase filter

   Input "Running Runs" becomes "running runs" after lowercase filter.

2. Step 2: Apply porter_stem filter

   Porter stemmer reduces "running" and "runs" to root "run" for both tokens.

3. Final Answer:

   ["run", "run"] -> Option D

4. Quick Check:

   Lowercase + stemmer = root forms in lowercase [OK]

Hint: Lowercase then stemmer gives root lowercase tokens [OK]

Common Mistakes:

• Ignoring stemming effect
• Not applying lowercase first
• Expecting original case tokens

4.

Identify the error in this token filter definition:

{
  "filter": {
    "my_stemmer": {
      "type": "stemmer",
      "language": "englishy"
    }
  }
}

medium

A. "type" should be "stemmer_override"

B. "language" value is misspelled; should be "english"

C. "filter" key must be inside "analyzer"

D. Missing "synonyms" array

Solution

1. Step 1: Check the "language" parameter spelling

   The language "englishy" is incorrect; the correct value is "english".

2. Step 2: Verify the stemmer type and parameters

   Type "stemmer" is correct; no synonyms needed here; filter can be defined outside analyzer.

3. Final Answer:

   "language" value is misspelled; should be "english" -> Option B

4. Quick Check:

   Stemmer language spelling must be exact [OK]

Hint: Check spelling of language names in stemmer filters [OK]

Common Mistakes:

• Using invalid language names
• Confusing stemmer with synonym filter
• Expecting synonyms in stemmer filter

5.

You want to create an analyzer that makes searches case-insensitive, finds root forms of words, and also matches synonyms like "fast" for "quick". Which token filters should you include in your analyzer?

hard

A. ["lowercase", "stemmer", "synonym"]

B. ["stop", "lowercase", "synonym"]

C. ["lowercase", "synonym"]

D. ["stemmer", "stop"]

Solution

1. Step 1: Identify filters for case-insensitivity and root forms

   Lowercase filter makes tokens lowercase; stemmer finds root forms.

2. Step 2: Add synonym filter for matching similar words

   Synonym filter adds synonyms like "fast" for "quick" to improve matching.

3. Final Answer:

   ["lowercase", "stemmer", "synonym"] -> Option A

4. Quick Check:

   Lowercase + stemmer + synonym = case-insensitive, root, and synonym matching [OK]

Hint: Combine lowercase, stemmer, and synonym filters for full matching [OK]

Common Mistakes:

• Omitting stemmer or synonym filters
• Using stop filter instead of stemmer
• Not including lowercase for case-insensitivity