Dashboard creation in Elasticsearch - Mini Project: Build & Apply

Dashboard creation

📖 Scenario: You are building a monitoring dashboard for a web application. Your Elasticsearch cluster has an index called web-logs containing request logs with fields like @timestamp, status_code, response_time, method, and endpoint. You need to create the aggregation queries that power each dashboard panel.

🎯 Goal: Write the Elasticsearch aggregation queries that would back a Kibana dashboard with four panels: total requests metric, status code breakdown, response time over time, and top endpoints table.

📋 What You'll Learn

Query for total request count in the last 24 hours

Aggregate status codes into a pie chart breakdown

Build a date histogram of average response time

Create a terms aggregation for top 10 endpoints by request count

💡 Why This Matters

🌍 Real World

Every Elasticsearch deployment uses Kibana dashboards for monitoring logs, application metrics, and business analytics in real time.

💼 Career

DevOps and SRE engineers build and maintain Kibana dashboards daily to monitor production systems and respond to incidents.

1

Total requests metric query

Write a query to the web-logs index that counts all documents in the last 24 hours. Use a range filter on @timestamp with gte set to "now-24h" and lte set to "now". Set size to 0 since we only need the count.

Elasticsearch

GET /web-logs/_search
{
  "size": 0,
  "query": {
    // Add range query on @timestamp for last 24 hours
    // Your code here
  }
}

Hint

Use a range query on @timestamp with relative date math: now-24h to now.

2

Status code breakdown aggregation

Extend the query to add an aggs section with a terms aggregation named "status_codes" on the field "status_code". Set size to 10 to get the top 10 status codes.

Elasticsearch

GET /web-logs/_search
{
  "size": 0,
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-24h",
        "lte": "now"
      }
    }
  },
  "aggs": {
    // Add terms aggregation on status_code field
    // Your code here
  }
}

Hint

Use a terms aggregation with the field set to status_code.

3

Response time over time histogram

Write a separate query with a date_histogram aggregation named "response_over_time" on "@timestamp" with a fixed_interval of "1h". Inside it, add a sub-aggregation named "avg_response" that computes the avg of the "response_time" field.

Elasticsearch

GET /web-logs/_search
{
  "size": 0,
  "aggs": {
    // Add date_histogram on @timestamp with 1h interval
    // Add avg sub-aggregation on response_time
    // Your code here
  }
}

Hint

Nest an avg aggregation inside the date_histogram to compute average response time per hour bucket.

4

Top endpoints table aggregation

Write a query with a terms aggregation named "top_endpoints" on "endpoint.keyword" with size 10. Add two sub-aggregations: "total_requests" using value_count on "endpoint.keyword", and "avg_time" using avg on "response_time".

Elasticsearch

GET /web-logs/_search
{
  "size": 0,
  "aggs": {
    // Add terms aggregation on endpoint.keyword
    // Add value_count and avg sub-aggregations
    // Your code here
  }
}

Hint

Use endpoint.keyword (not endpoint) for terms aggregation on a text field's keyword sub-field.

Practice

1. What is the main purpose of a dashboard in Elasticsearch's Kibana?

easy

A. To display multiple visualizations together for easy data analysis

B. To write complex Elasticsearch queries

C. To store raw data from Elasticsearch indexes

D. To manage user permissions for Elasticsearch

Solution

1. Step 1: Understand dashboard function

   A dashboard groups visualizations so users can see data insights in one place.

2. Step 2: Compare options

   Options A, B, and C describe other tasks not related to dashboard display.

3. Final Answer:

   To display multiple visualizations together for easy data analysis -> Option A

4. Quick Check:

   Dashboard = multiple visualizations [OK]

Hint: Dashboards show many visuals together for quick insights [OK]

Common Mistakes:

• Confusing dashboards with query writing
• Thinking dashboards store raw data
• Mixing dashboards with user management

2. Which syntax correctly adds a saved visualization to a Kibana dashboard?

easy

A. dashboard.addVisualization('vis_id')

B. dashboard.add('vis_id')

C. Dashboard.addVisualization('vis_id')

D. Dashboard.add('vis_id')

Solution

1. Step 1: Recall Kibana dashboard API

   The correct method to add a visualization is Dashboard.add('vis_id') with capital D.

2. Step 2: Check case sensitivity and method name

   dashboard.add('vis_id') uses lowercase dashboard object; options C and D use incorrect method name 'addVisualization'.

3. Final Answer:

   <code>Dashboard.add('vis_id')</code> -> Option D

4. Quick Check:

   Correct method is Dashboard.add() [OK]

Hint: Dashboard object is capitalized; method is add() [OK]

Common Mistakes:

• Using lowercase 'dashboard' instead of 'Dashboard'
• Using wrong method name like addVisualization
• Confusing method parameters

3. Given this Elasticsearch query used in a visualization:

{"query": {"match": {"status": "error"}}}

What will the visualization show when added to a dashboard?

medium

A. All documents with status 'error' count or details

B. All documents regardless of status

C. Documents with status 'success' only

D. An error message due to invalid query

Solution

1. Step 1: Analyze the query filter

   The query matches documents where the field 'status' equals 'error'.

2. Step 2: Understand visualization output

   The visualization will display data filtered to only those documents with status 'error'.

3. Final Answer:

   All documents with status 'error' count or details -> Option A

4. Quick Check:

   Query filters status='error' so visualization shows those docs [OK]

Hint: Match query filters data shown in visualization [OK]

Common Mistakes:

• Assuming it shows all documents
• Confusing 'error' with 'success'
• Thinking query syntax is invalid

4. You tried to add a visualization to a Kibana dashboard but it does not appear. Which is the most likely cause?

medium

A. The dashboard is already full and cannot add more visualizations

B. The Elasticsearch cluster is offline

C. The visualization ID used in the add command is incorrect

D. The visualization was created in a different tool

Solution

1. Step 1: Check visualization ID correctness

   If the ID is wrong, the dashboard cannot find and add the visualization.

2. Step 2: Evaluate other options

   Cluster offline would cause broader failures; dashboards do not have fixed limits; visualizations must be from Kibana.

3. Final Answer:

   The visualization ID used in the add command is incorrect -> Option C

4. Quick Check:

   Wrong ID means visualization won't load [OK]

Hint: Verify visualization ID matches exactly [OK]

Common Mistakes:

• Assuming dashboard has max visualization limit
• Ignoring ID typos
• Blaming Elasticsearch cluster without checking

5. You want to create a dashboard that shows error counts by hour and success counts by hour side by side. Which approach is best?

hard

A. Create a dashboard with only one visualization and switch filters manually

B. Create two visualizations with filters for 'error' and 'success', then add both to the dashboard

C. Create one visualization with a combined filter for 'error' and 'success' together

D. Create visualizations in different dashboards and link them

Solution

1. Step 1: Understand requirement for side-by-side comparison

   Two separate visualizations filtered by 'error' and 'success' allow clear side-by-side display.

2. Step 2: Evaluate other options

   Create one visualization with a combined filter for 'error' and 'success' together mixes filters, losing clarity; A requires manual switching; D separates data, not side-by-side.

3. Final Answer:

   Create two visualizations with filters for 'error' and 'success', then add both to the dashboard -> Option B

4. Quick Check:

   Separate filtered visuals show side-by-side data clearly [OK]

Hint: Use separate filtered visuals for clear side-by-side comparison [OK]

Common Mistakes:

• Combining filters in one visualization losing clarity
• Using one visualization and switching filters manually
• Splitting visuals across dashboards