Why Alerting and notifications in Elasticsearch? - Purpose & Use Cases

The Big Idea

What if your system could tell you about problems before your customers notice?

The Scenario

Imagine you run a busy online store and want to know immediately if the website goes down or if sales drop suddenly. Without alerting, you have to keep checking dashboards or logs all day and night.

The Problem

Manually watching data is tiring and easy to miss important problems. It wastes time and can cause delays in fixing issues, leading to unhappy customers and lost sales.

The Solution

Alerting and notifications automatically watch your data and send messages when something needs your attention. This means you get instant updates without constantly checking, so you can act fast.

Before vs After

✗ Before

Check logs every hour and email yourself if errors found

✓ After

Use Elasticsearch Watcher to send alerts instantly when errors occur

What It Enables

It lets you respond quickly to problems, keeping your system healthy and your users happy.

Real Life Example

A company uses Elasticsearch alerting to get notified immediately if their payment system fails, preventing lost transactions and customer frustration.

Key Takeaways

Manually monitoring data is slow and unreliable.

Alerting automates watching and notifying for important events.

This helps fix problems faster and improves service quality.

Practice

1. What is the main purpose of alerting in Elasticsearch?

easy

A. To automatically notify you when certain data conditions are met

B. To store large amounts of data efficiently

C. To visualize data in dashboards

D. To backup Elasticsearch indices

Solution

1. Step 1: Understand alerting concept

   Alerting watches your data and triggers notifications when specific conditions happen.

2. Step 2: Identify main purpose

   The main goal is to notify users automatically about important data changes or events.

3. Final Answer:

   To automatically notify you when certain data conditions are met -> Option A

4. Quick Check:

   Alerting = automatic notifications [OK]

Hint: Alerting means automatic notifications on data changes [OK]

Common Mistakes:

• Confusing alerting with data storage
• Thinking alerting is for data visualization
• Mixing alerting with backup processes

2. Which of the following is the correct syntax to define a trigger in an Elasticsearch alerting watch?

easy

A. "trigger": { "schedule": { "interval": "10m" } }

B. "trigger": "interval": "10m"

C. "trigger": { "interval": "10m" }

D. "trigger": { "time": "10m" }

Solution

1. Step 1: Recall trigger syntax in watch

   Triggers use a schedule object with an interval field inside curly braces.

2. Step 2: Match correct JSON structure

   "trigger": { "schedule": { "interval": "10m" } } correctly nests schedule and interval inside trigger with proper braces and quotes.

3. Final Answer:

   "trigger": { "schedule": { "interval": "10m" } } -> Option A

4. Quick Check:

   Trigger uses schedule with interval [OK]

Hint: Trigger syntax always nests schedule and interval inside braces [OK]

Common Mistakes:

• Missing braces around schedule
• Using wrong keys like 'time' instead of 'schedule'
• Incorrect JSON structure without nested objects

3. Given this watch input snippet, what type of input is being used?

{
  "input": {
    "search": {
      "request": {
        "indices": ["logs"],
        "body": {
          "query": { "match_all": {} }
        }
      }
    }
  }
}

medium

A. Webhook input

B. HTTP input

C. Search input

D. Script input

Solution

1. Step 1: Identify input type from JSON keys

   The input uses the key "search" with a request containing indices and a query.

2. Step 2: Match input type to Elasticsearch alerting inputs

   This matches the Search input type, which runs a search query on indices.

3. Final Answer:

   Search input -> Option C

4. Quick Check:

   Input with "search" key = Search input [OK]

Hint: Look for 'search' key to identify Search input type [OK]

Common Mistakes:

• Confusing search input with HTTP or webhook inputs
• Ignoring the 'search' key and guessing script input
• Not recognizing the query structure inside input

4. You wrote this action in your watch but it fails to send an email:

"actions": {
  "send_email": {
    "email": {
      "to": "[e-mail]",
      "subject": "Alert!",
      "body": "Condition met"
    }
  }
}

What is the likely error?

medium

A. Incorrect 'to' email format

B. Body must be an object, not a string

C. Missing 'trigger' section in watch

D. Missing 'from' field in email action

Solution

1. Step 1: Check required fields for email action

   Email action requires a 'from' field to specify sender address.

2. Step 2: Identify missing 'from' field

   The given action lacks the 'from' field, causing failure to send email.

3. Final Answer:

   Missing 'from' field in email action -> Option D

4. Quick Check:

   Email action needs 'from' field [OK]

Hint: Email actions always need a 'from' address [OK]

Common Mistakes:

• Assuming 'to' format is wrong when it is correct
• Forgetting to add 'from' sender email
• Thinking trigger absence causes email failure

5. You want to create an alert that sends a Slack message only if the number of errors in logs exceeds 100 in the last 5 minutes. Which condition correctly implements this in the watch?

hard

A. "condition": { "script": { "source": "return ctx.payload.hits.total.value > 100" } }

B. "condition": { "compare": { "ctx.payload.hits.total.value": { "gt": 100 } } }

C. "condition": { "script": { "source": "return ctx.payload.hits.total > 100" } }

D. "condition": { "compare": { "ctx.payload.hits.total": { "gte": 100 } } }

Solution

1. Step 1: Understand payload structure for hits total

   In Elasticsearch 7+, total hits count is accessed as ctx.payload.hits.total.value.

2. Step 2: Choose correct condition syntax

   The compare condition with 'gt' operator on ctx.payload.hits.total.value correctly checks if errors exceed 100.

3. Final Answer:

   "condition": { "compare": { "ctx.payload.hits.total.value": { "gt": 100 } } } -> Option B

4. Quick Check:

   Use compare with ctx.payload.hits.total.value > 100 [OK]

Hint: Use compare on ctx.payload.hits.total.value for counts [OK]

Common Mistakes:

• Using ctx.payload.hits.total instead of .value
• Using script with wrong field name
• Using 'gte' instead of 'gt' when strictly greater needed