Testing analyzers (_analyze API) in Elasticsearch - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the _analyze API in Elasticsearch?

The _analyze API helps you test how Elasticsearch breaks down text into tokens using a specific analyzer. It shows the tokens produced, their positions, and other details.

beginner

Which HTTP method is used to call the _analyze API?

You use the POST method to send text and analyzer settings to the _analyze API for testing.

intermediate

What key information does the _analyze API response provide?

It provides a list of tokens, each with its text, start and end offsets, position, and type. This helps you understand how text is processed.

intermediate

How can you specify a custom analyzer when using the _analyze API?

You include the analyzer name in the request body under the analyzer field, or define tokenizer and filters directly to test custom behavior.

beginner

Why is testing analyzers important before indexing data?

Testing analyzers ensures your text is tokenized as expected, which affects search accuracy and relevance. It helps catch issues early.

Which API helps you see how text is broken into tokens in Elasticsearch?

A _analyze API

B _search API

C _index API

D _mapping API

The _analyze API shows how text is tokenized by an analyzer.

What HTTP method do you use to test an analyzer with the _analyze API?

A GET

B PUT

C DELETE

D POST

You use POST to send text and analyzer settings to the _analyze API.

In the _analyze API response, what does each token include?

A Only the original text

B Text, position, offsets, and type

C Index name and document ID

D Search results count

Tokens include text, position, start and end offsets, and type.

How do you specify a custom analyzer in the _analyze API request?

A By setting the analyzer field in the request body

B By changing the index settings only

C By using the _search API

D By modifying the document mapping

You specify the analyzer in the request body to test it directly.

Why should you test analyzers before indexing data?

A To speed up indexing

B To delete old data

C To ensure text is tokenized correctly for accurate search

D To backup the index

Testing analyzers helps ensure search works well by tokenizing text properly.

Explain how the _analyze API helps you understand text processing in Elasticsearch.

Think about what happens to text before it is searchable.

Describe the steps to test a custom analyzer using the _analyze API.

Focus on how you send the request and what you look for in the response.

Practice

1. What is the main purpose of using the /_analyze API in Elasticsearch?

easy

A. To create a new index

B. To update the index mapping settings

C. To see how input text is broken into tokens by an analyzer

D. To delete documents from an index

Solution

1. Step 1: Understand the role of /_analyze

   The /_analyze API shows how text is processed by an analyzer, including tokenization and filtering.

2. Step 2: Compare with other API functions

   Updating mappings, deleting documents, or creating indexes are done by other APIs, not /_analyze.

3. Final Answer:

   To see how input text is broken into tokens by an analyzer -> Option C

4. Quick Check:

   Analyze API = tokenizes text [OK]

Hint: Remember: /_analyze shows token breakdown [OK]

Common Mistakes:

• Confusing /_analyze with index creation APIs
• Thinking it modifies data instead of analyzing text
• Assuming it deletes or updates documents

2. Which of the following is the correct syntax to test the standard analyzer on the text "Hello World" using the /_analyze API?

easy

A. { "analyzer": "standard", "text": "Hello World" }

B. { "analyzer": "standard" "text": "Hello World" }

C. { "text": "Hello World", "analyzer": standard }

D. { analyzer: "standard", text: "Hello World" }

Solution

1. Step 1: Check JSON syntax for /_analyze request body

   The body must be valid JSON with keys as strings and values properly quoted. { "analyzer": "standard", "text": "Hello World" } uses correct JSON syntax.

2. Step 2: Identify errors in other options

   { "analyzer": "standard" "text": "Hello World" } misses a comma between keys. { "text": "Hello World", "analyzer": standard } lacks quotes around "standard". { analyzer: "standard", text: "Hello World" } uses unquoted keys, invalid in JSON.

3. Final Answer:

   { "analyzer": "standard", "text": "Hello World" } -> Option A

4. Quick Check:

   Valid JSON with quoted keys and values = { "analyzer": "standard", "text": "Hello World" } [OK]

Hint: Always use quotes around keys and string values in JSON [OK]

Common Mistakes:

• Missing commas between JSON fields
• Not quoting string values
• Using unquoted keys in JSON

3. Given this request to /_analyze API with the standard analyzer:

{ "analyzer": "standard", "text": "Quick brown fox" }

What is the expected list of tokens returned?

medium

A. ["quick brown", "fox"]

B. ["quick", "brown", "fox"]

C. ["Quick", "brown", "fox"]

D. ["quick", "brown fox"]

Solution

1. Step 1: Understand standard analyzer behavior

   The standard analyzer lowercases text and splits on whitespace and punctuation, so "Quick brown fox" becomes tokens "quick", "brown", "fox".

2. Step 2: Check options for token case and splitting

   ["quick", "brown", "fox"] shows all lowercase separate tokens. ["Quick", "brown", "fox"] keeps uppercase 'Quick' which standard analyzer lowercases. Options that combine words are incorrect.

3. Final Answer:

   ["quick", "brown", "fox"] -> Option B

4. Quick Check:

   Standard analyzer lowercases and splits words [OK]

Hint: Standard analyzer lowercases and splits on spaces [OK]

Common Mistakes:

• Expecting tokens to keep original case
• Thinking tokens combine multiple words
• Confusing analyzer output with raw text

4. You run this /_analyze request:

{ "analyzer": "whitespace", "text": "Hello, World!" }

But the output tokens include punctuation marks like "Hello," instead of just "Hello". What is the likely cause?

medium

A. Using the whitespace analyzer which only splits on spaces, not removing punctuation

B. The standard analyzer was used instead of whitespace

C. The text was not provided in the request body

D. The analyzer automatically removes punctuation by default

Solution

1. Step 1: Recall whitespace analyzer behavior

   The whitespace analyzer splits text only on spaces and does not remove punctuation, so tokens keep punctuation marks.

2. Step 2: Eliminate other options

   The standard analyzer was used instead of whitespace is incorrect because the standard analyzer removes punctuation. The text was not provided in the request body is invalid because text is provided. The analyzer automatically removes punctuation by default is false; whitespace analyzer does not remove punctuation.

3. Final Answer:

   Using the whitespace analyzer which only splits on spaces, not removing punctuation -> Option A

4. Quick Check:

   Whitespace analyzer keeps punctuation in tokens [OK]

Hint: Whitespace analyzer splits only on spaces, keeps punctuation [OK]

Common Mistakes:

• Expecting punctuation removal with whitespace analyzer
• Confusing analyzer names
• Omitting text in request body

5. You want to test a custom analyzer named "my_custom_analyzer" that uses a lowercase filter and a stopword filter removing "the" and "and". Which /_analyze request body will correctly show tokens for the text "The quick brown fox and the lazy dog"?

hard

A. { "analyzer": "standard", "filter": ["lowercase", "stop"], "text": "The quick brown fox and the lazy dog" }

B. { "analyzer": "my_custom_analyzer", "text": "The quick brown fox and the lazy dog", "stopwords": ["the", "and"] }

C. { "tokenizer": "standard", "filter": ["lowercase", "stop"], "text": "The quick brown fox and the lazy dog" }

D. { "analyzer": "my_custom_analyzer", "text": "The quick brown fox and the lazy dog" }

Solution

1. Step 1: Use the custom analyzer name in the /_analyze request

   To test a custom analyzer, specify its name under the "analyzer" key with the text to analyze. { "analyzer": "my_custom_analyzer", "text": "The quick brown fox and the lazy dog" } does this correctly.

2. Step 2: Understand why other options are incorrect

   { "analyzer": "my_custom_analyzer", "text": "The quick brown fox and the lazy dog", "stopwords": ["the", "and"] } incorrectly tries to add stopwords in the request body, which is not supported here. { "tokenizer": "standard", "filter": ["lowercase", "stop"], "text": "The quick brown fox and the lazy dog" } tries to define tokenizer and filters inline, which is invalid for /_analyze without full analyzer definition. { "analyzer": "standard", "filter": ["lowercase", "stop"], "text": "The quick brown fox and the lazy dog" } uses standard analyzer, not the custom one.

3. Final Answer:

   { "analyzer": "my_custom_analyzer", "text": "The quick brown fox and the lazy dog" } -> Option D

4. Quick Check:

   Use custom analyzer name directly in /_analyze [OK]

Hint: Pass custom analyzer name directly to /_analyze [OK]

Common Mistakes:

• Trying to redefine filters in /_analyze request
• Using standard analyzer instead of custom
• Adding unsupported keys like stopwords in request body