Resource monitoring best practices in Kubernetes - Step-by-Step Execution

Process Flow - Resource monitoring best practices

Start Monitoring Setup

↓

Define Metrics to Monitor

↓

Deploy Monitoring Tools

↓

Collect Resource Usage Data

↓

Analyze Data & Set Alerts

↓

Optimize Resources Based on Insights

↓

Repeat Cycle

This flow shows the steps to set up and maintain resource monitoring in Kubernetes, from defining metrics to optimizing resources.

Execution Sample

Kubernetes

kubectl top pods
kubectl top nodes
kubectl describe pod <pod-name>
# Set up Prometheus and Alertmanager
# Define CPU and memory thresholds
# Create alerts for high usage

Commands and steps to monitor resource usage and set alerts in Kubernetes.

Process Table

Step	Action	Command/Tool	Result/Output
1	Check pod resource usage	kubectl top pods	Shows CPU and memory usage per pod
2	Check node resource usage	kubectl top nodes	Shows CPU and memory usage per node
3	Inspect pod details	kubectl describe pod <pod-name>	Shows detailed pod resource requests and limits
4	Deploy monitoring stack	Install Prometheus and Alertmanager	Prometheus collects metrics, Alertmanager handles alerts
5	Set alert rules	Configure CPU/memory thresholds	Alerts trigger when usage exceeds thresholds
6	Analyze alerts	Review alert notifications	Identify resource bottlenecks or leaks
7	Optimize resources	Adjust pod requests/limits or scale	Improved resource utilization and stability
8	Repeat monitoring cycle	Continuous monitoring	Ongoing resource health and performance

💡 Monitoring cycle repeats continuously to maintain cluster health

Status Tracker

Variable	Start	After Step 3	After Step 5	After Step 7	Final
CPU Usage	Unknown	Observed per pod	Thresholds set	Optimized	Stable
Memory Usage	Unknown	Observed per pod	Thresholds set	Optimized	Stable
Alerts	None	None	Configured	Triggered and handled	Managed
Resource Requests/Limits	Default or unset	Checked	Reviewed	Adjusted	Appropriate

Key Moments - 3 Insights

Why do we need to set resource requests and limits for pods?

Setting requests and limits ensures pods get enough resources and prevents one pod from using too much, as seen in step 3 and step 7 where pod details are checked and then optimized.

What happens if we don’t set alerts for resource usage?

Without alerts (step 5), high resource usage might go unnoticed, causing performance issues or crashes, which is why alerting is critical for timely action.

Why is monitoring a continuous cycle?

Resource needs change over time; continuous monitoring (step 8) helps catch new issues early and keeps the cluster healthy by repeating the process.

Visual Quiz - 3 Questions

Look at the execution table, what command shows detailed pod resource requests and limits?

A kubectl get pods

B kubectl top nodes

C kubectl describe pod <pod-name>

D kubectl logs <pod-name>

💡 Hint

Check step 3 in the execution table where pod details are inspected.

At which step are alerts configured for CPU and memory usage?

A Step 2

B Step 5

C Step 7

D Step 1

💡 Hint

Look at the execution table row describing alert rule setup.

If resource requests are not adjusted after monitoring, what is likely to happen?

A Resource bottlenecks may persist

B Resource usage remains optimized

C Alerts will never trigger

D Pods will automatically scale

💡 Hint

Refer to step 7 and variable tracker where optimization is key to fixing bottlenecks.

Concept Snapshot

Resource Monitoring Best Practices in Kubernetes:
- Use 'kubectl top' to check pod and node usage
- Set resource requests and limits for pods
- Deploy Prometheus and Alertmanager for metrics and alerts
- Define CPU and memory thresholds for alerts
- Analyze alerts and optimize resources regularly
- Repeat monitoring continuously for cluster health

Full Transcript

This visual execution guide shows how to monitor resources in Kubernetes effectively. First, you check resource usage of pods and nodes using 'kubectl top'. Then, inspect pod resource requests and limits with 'kubectl describe pod'. Next, deploy monitoring tools like Prometheus and Alertmanager to collect metrics and send alerts. Set alert rules for CPU and memory usage thresholds. When alerts trigger, analyze them to find resource bottlenecks. Adjust pod resource requests and limits or scale pods to optimize usage. This process repeats continuously to keep the cluster healthy and stable.

Practice

1. Why is it important to set resource requests and limits in Kubernetes pods?

easy

A. To ensure pods get the resources they need and prevent resource conflicts

B. To make pods run slower and use more CPU

C. To disable monitoring tools automatically

D. To allow unlimited resource usage without restrictions

Solution

1. Step 1: Understand resource requests and limits

   Resource requests define the minimum resources a pod needs, and limits set the maximum it can use.

2. Step 2: Recognize the effect on cluster stability

   Setting these prevents pods from using too many resources and causing conflicts or crashes.

3. Final Answer:

   To ensure pods get the resources they need and prevent resource conflicts -> Option A

4. Quick Check:

   Resource requests and limits = prevent conflicts [OK]

Hint: Requests = minimum, limits = maximum resources [OK]

Common Mistakes:

• Thinking limits slow down pods intentionally
• Believing requests disable monitoring
• Assuming unlimited usage is safe

2. Which command correctly shows current CPU and memory usage of pods in Kubernetes?

easy

A. kubectl monitor pods

B. kubectl get pods --usage

C. kubectl top pods

D. kubectl describe pods --metrics

Solution

1. Step 1: Identify the command for resource usage

   The kubectl top pods command shows CPU and memory usage of pods.

2. Step 2: Check other options for correctness

   Other commands are invalid or do not show usage metrics.

3. Final Answer:

   kubectl top pods -> Option C

4. Quick Check:

   Usage command = kubectl top pods [OK]

Hint: Use 'kubectl top pods' to see pod resource usage [OK]

Common Mistakes:

• Using 'kubectl get pods --usage' which is invalid
• Confusing 'describe' with usage metrics
• Assuming 'kubectl monitor' is a valid command

3. Given this command output:

NAME          CPU(cores)   MEMORY(bytes)
myapp-pod-1   150m         200Mi
myapp-pod-2   300m         400Mi

What is the total CPU usage of both pods?

medium

A. 300m

B. 450m

C. 150m

D. 600m

Solution

1. Step 1: Add CPU usage values from both pods

   150m + 300m = 450m CPU cores.

2. Step 2: Confirm units and sum

   Both values are in millicores (m), so sum is 450m.

3. Final Answer:

   450m -> Option B

4. Quick Check:

   150m + 300m = 450m [OK]

Hint: Add CPU millicores values directly [OK]

Common Mistakes:

• Adding memory values instead of CPU
• Confusing 450m with 600m
• Ignoring units and summing incorrectly

4. You set resource limits on a pod, but kubectl top pods shows usage exceeding those limits. What is the likely cause?

medium

A. The pod has no resource requests set

B. The pod is using burstable QoS and can exceed limits temporarily

C. Resource limits are not enforced by Kubernetes by default

D. The metrics server is not installed or reporting incorrect data

Solution

1. Step 1: Understand resource limits enforcement

   Kubernetes enforces limits strictly; pods cannot exceed set limits.

2. Step 2: Consider metrics server role

   If usage shows above limits, metrics server may be missing or reporting wrong data.

3. Final Answer:

   The metrics server is not installed or reporting incorrect data -> Option D

4. Quick Check:

   Incorrect metrics = wrong usage shown [OK]

Hint: Check metrics server if usage exceeds limits [OK]

Common Mistakes:

• Thinking Kubernetes allows exceeding limits
• Confusing QoS classes with limit enforcement
• Ignoring metrics server installation

5. You want to monitor resource usage trends over time for your Kubernetes cluster. Which approach is best?

hard

A. Set resource requests and limits, then use a monitoring tool like Prometheus

B. Use kubectl top repeatedly and save output manually

C. Only set resource limits without monitoring tools

D. Rely on kubectl describe to check resource usage daily

Solution

1. Step 1: Understand monitoring needs over time

   Manual commands show current usage but not trends or history.

2. Step 2: Use monitoring tools with resource limits

   Setting requests/limits ensures stable usage; tools like Prometheus collect and visualize trends.

3. Final Answer:

   Set resource requests and limits, then use a monitoring tool like Prometheus -> Option A

4. Quick Check:

   Requests + monitoring tool = best practice [OK]

Hint: Combine limits with Prometheus for trend monitoring [OK]

Common Mistakes:

• Relying on manual commands for long-term trends
• Skipping resource requests or limits
• Using describe command for usage stats