Infrastructure monitoring in Elasticsearch - Mini Project: Build & Apply

Infrastructure Monitoring with Elasticsearch

📖 Scenario: You are setting up a simple infrastructure monitoring system using Elasticsearch. You want to store server metrics like CPU and memory usage, then query them to find servers with high CPU usage.

🎯 Goal: Build an Elasticsearch index with server metrics, configure a threshold for high CPU usage, query the index to find servers exceeding that threshold, and display the results.

📋 What You'll Learn

Create an Elasticsearch index called server_metrics with sample server data

Add a variable cpu_threshold to set the CPU usage limit

Write a query to find servers with CPU usage greater than cpu_threshold

Print the names of servers exceeding the CPU threshold

💡 Why This Matters

🌍 Real World

Monitoring server health and performance is critical in IT operations to prevent downtime and optimize resources.

💼 Career

DevOps engineers and system administrators use Elasticsearch to collect, query, and analyze infrastructure metrics for proactive monitoring.

1

Create the server_metrics index with sample data

Create an Elasticsearch index called server_metrics and add these exact documents: {"server": "server1", "cpu": 55, "memory": 70}, {"server": "server2", "cpu": 85, "memory": 60}, {"server": "server3", "cpu": 40, "memory": 80}.

Elasticsearch

# Use the Elasticsearch bulk API to create the index and add documents
# Your code here

Need a hint?

Use the Elasticsearch bulk API format to add multiple documents to the server_metrics index.

2

Set the CPU usage threshold variable

Create a variable called cpu_threshold and set it to 70 to represent the CPU usage limit.

Elasticsearch

POST /server_metrics/_bulk
{ "index": {} }
{ "server": "server1", "cpu": 55, "memory": 70 }
{ "index": {} }
{ "server": "server2", "cpu": 85, "memory": 60 }
{ "index": {} }
{ "server": "server3", "cpu": 40, "memory": 80 }

# Create cpu_threshold variable
# Your code here

Need a hint?

Use an Elasticsearch stored script or a variable in your query to represent the CPU threshold of 70.

3

Query servers with CPU usage above cpu_threshold

Write an Elasticsearch query to find all documents in server_metrics where the cpu field is greater than 70.

Elasticsearch

POST /server_metrics/_bulk
{ "index": {} }
{ "server": "server1", "cpu": 55, "memory": 70 }
{ "index": {} }
{ "server": "server2", "cpu": 85, "memory": 60 }
{ "index": {} }
{ "server": "server3", "cpu": 40, "memory": 80 }

PUT /_scripts/cpu_threshold
{
  "script": {
    "lang": "painless",
    "source": "return 70"
  }
}

POST /server_metrics/_search
{
  "query": {
    # Your code here
  }
}

Need a hint?

Use a range query on the cpu field with gt set to 70.

4

Display the names of servers exceeding the CPU threshold

Print the server names from the query results where CPU usage is greater than 70.

Elasticsearch

POST /server_metrics/_bulk
{ "index": {} }
{ "server": "server1", "cpu": 55, "memory": 70 }
{ "index": {} }
{ "server": "server2", "cpu": 85, "memory": 60 }
{ "index": {} }
{ "server": "server3", "cpu": 40, "memory": 80 }

PUT /_scripts/cpu_threshold
{
  "script": {
    "lang": "painless",
    "source": "return 70"
  }
}

POST /server_metrics/_search
{
  "query": {
    "range": {
      "cpu": {
        "gt": 70
      }
    }
  }
}

# Print the server names from the search hits
# Your code here

Need a hint?

Look at the hits.hits array in the search response and print the _source.server field for each hit.