Application performance monitoring in Elasticsearch - Mini Project: Build & Apply

Application Performance Monitoring with Elasticsearch

📖 Scenario: You are a DevOps engineer responsible for monitoring the performance of a web application. You want to collect and analyze response times of different API endpoints to identify slow requests.

🎯 Goal: Build a simple Elasticsearch query to filter and aggregate API response times for performance monitoring.

📋 What You'll Learn

Create an Elasticsearch index mapping for API response data

Add a filter to select only requests with response time greater than a threshold

Aggregate average response time per API endpoint

Display the aggregation results

💡 Why This Matters

🌍 Real World

Monitoring API response times helps detect performance issues early and improve user experience.

💼 Career

DevOps engineers use Elasticsearch queries and aggregations to analyze logs and metrics for application performance monitoring.

1

Create the Elasticsearch index mapping

Create an Elasticsearch index mapping called api_performance with fields: endpoint as keyword and response_time_ms as integer.

Elasticsearch

# Define the index mapping for api_performance
# Your code here

Hint

Use the PUT method to create the index with the specified mapping.

2

Add a filter for slow requests

Write an Elasticsearch query that filters documents in api_performance where response_time_ms is greater than 200 milliseconds.

Elasticsearch

PUT /api_performance
{
  "mappings": {
    "properties": {
      "endpoint": { "type": "keyword" },
      "response_time_ms": { "type": "integer" }
    }
  }
}

# Add a query filter for response_time_ms > 200
# Your code here

Hint

Use a range query to filter documents where response_time_ms is greater than 200.

3

Aggregate average response time per endpoint

Extend the query to include an aggregation named avg_response_time that calculates the average response_time_ms grouped by endpoint.

Elasticsearch

PUT /api_performance
{
  "mappings": {
    "properties": {
      "endpoint": { "type": "keyword" },
      "response_time_ms": { "type": "integer" }
    }
  }
}

GET /api_performance/_search
{
  "query": {
    "range": {
      "response_time_ms": {
        "gt": 200
      }
    }
  },
  "aggs": {
    "avg_response_time": {
      "terms": {
        "field": "endpoint"
      },
      "aggs": {
        "average_time": {
          "avg": {
            "field": "response_time_ms"
          }
        }
      }
    }
  }
}

# Your code here

Hint

Use a terms aggregation on endpoint and nest an avg aggregation on response_time_ms.

4

Display the aggregation results

Run the query and write the expected output format showing average response times per endpoint as JSON.

Elasticsearch

PUT /api_performance
{
  "mappings": {
    "properties": {
      "endpoint": { "type": "keyword" },
      "response_time_ms": { "type": "integer" }
    }
  }
}

GET /api_performance/_search
{
  "query": {
    "range": {
      "response_time_ms": {
        "gt": 200
      }
    }
  },
  "aggs": {
    "avg_response_time": {
      "terms": {
        "field": "endpoint"
      },
      "aggs": {
        "average_time": {
          "avg": {
            "field": "response_time_ms"
          }
        }
      }
    }
  }
}

# Print the aggregation buckets with endpoint and average_time.value
# Your code here

Hint

The output shows buckets with each endpoint and its average response time value.

Practice

1. What is the main purpose of Application Performance Monitoring (APM) in Elasticsearch?

easy

A. To track application speed and detect errors

B. To store user login credentials securely

C. To manage Elasticsearch cluster nodes

D. To backup Elasticsearch indexes automatically

Solution

1. Step 1: Understand APM's role

   APM is designed to monitor how fast an application runs and to find any errors it produces.

2. Step 2: Match purpose with options

   Only To track application speed and detect errors describes tracking speed and errors, which fits APM's main goal.

3. Final Answer:

   To track application speed and detect errors -> Option A

4. Quick Check:

   APM purpose = Track speed and errors [OK]

Hint: APM = watch app speed and errors [OK]

Common Mistakes:

• Confusing APM with security or backup tools
• Thinking APM manages cluster nodes
• Assuming APM stores user credentials

2. Which Elasticsearch query syntax correctly calculates the average response time from APM data?

easy

A. POST /apm-*/_update_by_query {"script": {"source": "ctx._source.duration = 0"}}

B. GET /apm-*/_search {"query": {"match_all": {}}, "size":10}

C. GET /apm-*/_search {"size":0, "aggs": {"avg_response_time": {"avg": {"field": "transaction.duration.us"}}}}

D. GET /apm-*/_search {"aggs": {"max_response_time": {"max": {"field": "transaction.duration.us"}}}}

Solution

1. Step 1: Identify aggregation for average

   The query uses "avg" aggregation on the field "transaction.duration.us" which stores response times in microseconds.

2. Step 2: Confirm query structure

   Size is 0 to avoid returning documents, focusing only on aggregation results, which is correct for average calculation.

3. Final Answer:

   GET /apm-*/_search {"size":0, "aggs": {"avg_response_time": {"avg": {"field": "transaction.duration.us"}}}} -> Option C

4. Quick Check:

   Average aggregation query = GET /apm-*/_search {"size":0, "aggs": {"avg_response_time": {"avg": {"field": "transaction.duration.us"}}}} [OK]

Hint: Average uses "avg" aggregation with size 0 [OK]

Common Mistakes:

• Using match_all without aggregation
• Using update_by_query instead of search
• Using max aggregation instead of avg

3. Given this Elasticsearch aggregation query on APM data, what is the expected output?

GET /apm-*/_search
{
  "size": 0,
  "aggs": {
    "avg_response_time": {
      "avg": { "field": "transaction.duration.us" }
    }
  }
}

medium

A. {"aggregations":{"max_response_time":{"value":500000}}}

B. {"hits":{"total":100}}

C. {"error":"Field not found"}

D. {"aggregations":{"avg_response_time":{"value":250000}}}

Solution

1. Step 1: Understand aggregation type

   The query requests the average of the field "transaction.duration.us" which holds response times in microseconds.

2. Step 2: Match output to aggregation

   The output shows an aggregation named "avg_response_time" with a numeric value representing the average, matching {"aggregations":{"avg_response_time":{"value":250000}}}.

3. Final Answer:

   {"aggregations":{"avg_response_time":{"value":250000}}} -> Option D

4. Quick Check:

   Average aggregation output = {"aggregations":{"avg_response_time":{"value":250000}}} [OK]

Hint: Aggregation output shows "aggregations" with average value [OK]

Common Mistakes:

• Confusing hits total with aggregation result
• Expecting max instead of avg
• Assuming error without checking field existence

4. You run this Elasticsearch query to get average response time but get an error: Fielddata is disabled on text fields by default. What is the likely cause?

medium

A. Trying to aggregate on a text field instead of a numeric field

B. Using the wrong index pattern in the query

C. Missing authentication credentials

D. Query syntax error in aggregation block

Solution

1. Step 1: Analyze error message

   The error says fielddata is disabled on text fields, which means aggregation was attempted on a text field.

2. Step 2: Understand aggregation requirements

   Aggregations like average require numeric fields, so using a text field causes this error.

3. Final Answer:

   Trying to aggregate on a text field instead of a numeric field -> Option A

4. Quick Check:

   Fielddata error = Aggregation on text field [OK]

Hint: Average needs numeric field, not text [OK]

Common Mistakes:

• Blaming index pattern or auth for this error
• Assuming syntax error without checking field type
• Ignoring field data type requirements

5. You want to monitor the average response time for your app but only for transactions with errors. Which Elasticsearch query snippet correctly filters and calculates this?

hard

A. { "size": 0, "query": { "term": { "error.id": "" } }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } }

B. { "size": 0, "query": { "exists": { "field": "error.id" } }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } }

C. { "size": 0, "query": { "match_all": {} }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } }

D. { "size": 0, "query": { "term": { "transaction.status": "success" } }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } }

Solution

1. Step 1: Identify filter for transactions with errors

   Transactions with errors have a non-empty "error.id" field, so we use "exists" query on "error.id".

2. Step 2: Confirm aggregation on filtered data

   The aggregation calculates average response time only on filtered documents, which is correct.

3. Final Answer:

   { "size": 0, "query": { "exists": { "field": "error.id" } }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } } -> Option B

4. Quick Check:

   Filter errors with exists + avg aggregation = { "size": 0, "query": { "exists": { "field": "error.id" } }, "aggs": { "avg_response_time": { "avg": { "field": "transaction.duration.us" } } } } [OK]

Hint: Use exists query to filter error transactions [OK]

Common Mistakes:

• Using empty term query instead of exists
• Calculating average without filtering errors
• Filtering for success instead of errors