Default vs custom metrics in AWS - Trade-offs & Expert Analysis

Overview - Default vs custom metrics

What is it?

Metrics are measurements that tell you how well your cloud resources are working. Default metrics are automatically collected by cloud services without extra setup. Custom metrics are measurements you create yourself to track things specific to your needs. Both help you understand and improve your cloud systems.

Why it matters

Without metrics, you would be guessing if your cloud resources are healthy or efficient. Default metrics give basic health info, but they might miss important details unique to your app. Custom metrics fill that gap, helping you catch problems early and optimize performance. Without them, outages or slowdowns could go unnoticed until users complain.

Where it fits

Before learning this, you should understand basic cloud services and monitoring concepts. After this, you can learn how to set alarms and automate responses based on metrics. This knowledge fits into managing cloud infrastructure and ensuring reliable applications.

Mental Model

Core Idea

Metrics are like the vital signs of your cloud resources, with default metrics as the basic checkup and custom metrics as specialized tests you choose.

Think of it like...

Imagine going to a doctor: default metrics are like standard blood pressure and temperature checks done for everyone, while custom metrics are like specific allergy tests or scans tailored to your unique health needs.

┌───────────────┐       ┌───────────────┐
│ Default       │       │ Custom        │
│ Metrics      │       │ Metrics       │
│ (Auto-collected)│       │ (User-defined) │
└──────┬────────┘       └──────┬────────┘
       │                       │
       │                       │
       ▼                       ▼
  Basic health info       Specific insights
       │                       │
       └────────────┬──────────┘
                    ▼
             Cloud Monitoring

Build-Up - 7 Steps

1

Foundation

What are metrics in cloud monitoring

Concept: Introduce the basic idea of metrics as measurements of cloud resource behavior.

Metrics are numbers that tell you how your cloud resources like servers, databases, or applications are performing. For example, CPU usage or network traffic. They help you see if things are working well or if there are problems.

Result

You understand that metrics are essential data points for monitoring cloud systems.

Knowing what metrics are is the foundation for all cloud monitoring and management.

2

Foundation

Understanding default metrics

Concept: Explain default metrics as those automatically provided by cloud services.

Cloud providers like AWS automatically collect common metrics for their services. For example, EC2 instances report CPU utilization and disk reads without you doing anything. These are called default metrics because they come ready-made.

Result

You can identify default metrics and know they require no extra setup.

Recognizing default metrics helps you quickly get basic health info without extra work.

3

Intermediate

Why create custom metrics

🤔Before reading on: do you think default metrics cover all monitoring needs or only basic ones? Commit to your answer.

Concept: Introduce the need for custom metrics to track specific application or business data.

Default metrics cover general system health but might miss details important to your app. Custom metrics let you measure things like user logins, order counts, or error rates that default metrics don't track. You send these metrics to the cloud monitoring service yourself.

Result

You understand the purpose and value of custom metrics.

Knowing why custom metrics exist helps you design better monitoring tailored to your unique needs.

4

Intermediate

How to create and send custom metrics

🤔Before reading on: do you think custom metrics are automatically collected or require manual setup? Commit to your answer.

Concept: Explain the process of defining and sending custom metrics to the cloud monitoring system.

To create custom metrics, you write code or use tools to send your specific data points to the cloud monitoring service. For example, in AWS CloudWatch, you use the PutMetricData API to send your custom metric values regularly.

Result

You know the practical steps to implement custom metrics.

Understanding how to send custom metrics empowers you to monitor what matters most.

5

Intermediate

Comparing default and custom metrics features

🤔Before reading on: do you think default and custom metrics have the same cost and retention policies? Commit to your answer.

Concept: Highlight differences in cost, retention, and granularity between default and custom metrics.

Default metrics are usually free or low cost and retained for a standard period. Custom metrics often cost more and you decide how long to keep them. Also, custom metrics can have finer detail or track unique data, while default metrics are fixed.

Result

You can weigh tradeoffs when choosing metrics to monitor.

Knowing differences helps optimize monitoring costs and effectiveness.

6

Advanced

Using custom metrics for proactive alerting

🤔Before reading on: do you think alerts can be based only on default metrics or also on custom metrics? Commit to your answer.

Concept: Show how custom metrics enable precise alerts for business or app-specific conditions.

You can set alarms that trigger when custom metrics cross thresholds, like too many failed logins or slow response times. This helps catch issues before users notice. Default metrics alone might not detect these specific problems.

Result

You see how custom metrics improve operational awareness.

Understanding alerting with custom metrics enables proactive system management.

7

Expert

Challenges and best practices with custom metrics

🤔Before reading on: do you think sending many custom metrics can impact system performance or cost? Commit to your answer.

Concept: Discuss performance, cost, and design considerations when using custom metrics at scale.

Sending too many custom metrics can increase costs and add overhead to your systems. Best practices include batching metric data, choosing meaningful metrics, and cleaning up unused ones. Also, consider metric naming conventions and units for clarity.

Result

You learn how to use custom metrics efficiently and avoid pitfalls.

Knowing these challenges helps maintain scalable and cost-effective monitoring.

Under the Hood

Cloud monitoring systems collect metrics by gathering data points from resources at regular intervals. Default metrics come from built-in agents or service APIs that automatically report common stats. Custom metrics require your code or tools to send data via APIs to the monitoring backend, where it is stored, aggregated, and made queryable.

Why designed this way?

Default metrics were designed to provide quick, easy health checks without user effort. Custom metrics were added later to allow flexibility for diverse applications and business needs. This separation balances ease of use with customization, avoiding overload of irrelevant data.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Cloud Service │──────▶│ Default       │       │ Custom        │
│ Resources    │       │ Metrics       │       │ Metrics       │
│ (EC2, RDS)   │       │ (Auto-collected)│       │ (User-sent)   │
└──────┬────────┘       └──────┬────────┘       └──────┬────────┘
       │                       │                       │
       │                       │                       │
       ▼                       ▼                       ▼
  Metrics Data           Metrics Data           Metrics Data
  Collection             Storage & Query        Storage & Query
       │                       │                       │
       └───────────────┬───────┴───────────────┬───────┘
                       ▼                       ▼
                 Monitoring Dashboard & Alerts

Myth Busters - 4 Common Misconceptions

Quick: do you think default metrics cover all monitoring needs? Commit yes or no.

Common Belief: Default metrics are enough to monitor any cloud application fully.

Reality: Default metrics provide only basic system health data and often miss application-specific or business-related insights.

Why it matters: Relying only on default metrics can cause missed problems or slow reactions to issues unique to your app.

Quick: do you think custom metrics are free like default metrics? Commit yes or no.

Common Belief: Custom metrics cost the same as default metrics or are free.

Reality: Custom metrics usually incur additional costs based on volume and retention time, unlike many default metrics which are free or cheaper.

Why it matters: Ignoring cost differences can lead to unexpected bills and budget overruns.

Quick: do you think custom metrics are collected automatically? Commit yes or no.

Common Belief: Custom metrics are automatically collected by the cloud provider like default metrics.

Reality: Custom metrics must be explicitly sent by your code or tools; they are not collected automatically.

Why it matters: Assuming automatic collection can cause missing critical data and blind spots in monitoring.

Quick: do you think sending many custom metrics has no impact on system performance? Commit yes or no.

Common Belief: Sending many custom metrics does not affect system performance or cost significantly.

Reality: High volume of custom metrics can increase network load, processing overhead, and costs.

Why it matters: Not managing custom metrics volume can degrade system performance and inflate expenses.

Expert Zone

1

Custom metrics require careful naming conventions and units to ensure clarity and avoid confusion in dashboards and alerts.

2

Batching metric data points before sending reduces API calls and improves performance and cost efficiency.

3

Retention policies for custom metrics should balance historical insight needs with storage costs, often requiring lifecycle management.

When NOT to use

Avoid custom metrics when default metrics already provide sufficient insight or when the cost and complexity outweigh the benefits. Instead, use enhanced default metrics or logs for troubleshooting.

Production Patterns

In production, teams use custom metrics to track business KPIs like transaction rates or user engagement alongside default system metrics. They integrate these metrics with alerting and automated scaling to maintain reliability and performance.

Connections

Application Performance Monitoring (APM)

Builds-on

Understanding metrics is foundational to APM tools that combine metrics, traces, and logs for full visibility.

Financial Budgeting

Analogy

Just like tracking expenses helps manage budgets, monitoring metrics helps manage cloud resource costs and performance.

Human Vital Signs Monitoring

Similar pattern

Both track key indicators to detect early signs of problems and maintain health, whether in humans or cloud systems.

Common Pitfalls

#1 Assuming default metrics cover all monitoring needs.

Wrong approach:

Relying only on CPU and memory usage metrics to detect application errors.

Correct approach:

Implement custom metrics to track application-specific errors and business events.

Root cause:

Misunderstanding the scope and limitations of default metrics.

#2 Sending custom metrics too frequently without batching.

Wrong approach:

Calling PutMetricData API for every single event immediately.

Correct approach:

Batch multiple metric data points and send them together periodically.

Root cause:

Not considering API rate limits and performance impact.

#3 Ignoring cost implications of custom metrics.

Wrong approach:

Creating hundreds of custom metrics without monitoring usage or cost.

Correct approach:

Plan and monitor custom metric volume and retention to control costs.

Root cause:

Lack of awareness about pricing models for custom metrics.

Key Takeaways

Default metrics provide automatic, basic health data for cloud resources but may miss application-specific details.

Custom metrics let you track unique, important data tailored to your application's needs, improving monitoring precision.

Creating custom metrics requires manual setup and careful management to balance insight, performance, and cost.

Effective monitoring combines both default and custom metrics to maintain cloud system health and business success.

Understanding the differences and best practices around metrics empowers proactive, cost-effective cloud management.

Practice

1. Which statement best describes AWS default metrics in CloudWatch?

easy

A. They are automatically collected and free within certain limits.

B. They require manual setup and extra charges.

C. They only track custom application data.

D. They are disabled by default and need activation.

Solution

1. Step 1: Understand default metrics collection

   AWS CloudWatch automatically collects default metrics for many services without extra setup.

2. Step 2: Recognize cost and availability

   These default metrics are free within certain usage limits, making them easy to use.

3. Final Answer:

   They are automatically collected and free within certain limits. -> Option A

4. Quick Check:

   Default metrics = automatic + free [OK]

Hint: Default metrics are automatic and free within limits [OK]

Common Mistakes:

• Thinking default metrics need manual setup
• Assuming default metrics track custom data
• Believing default metrics are disabled by default

2. Which of the following is the correct way to send a custom metric to AWS CloudWatch using the AWS CLI?

easy

A. aws cloudwatch send-metric --name CPUUtilization --value 70

B. aws cloudwatch add-metric-data --metric-name CPUUtilization --value 70

C. aws cloudwatch create-metric --metric CPUUtilization --value 70

D. aws cloudwatch put-metric-data --metric-name CPUUtilization --namespace AWS/EC2 --value 70

Solution

1. Step 1: Identify the correct AWS CLI command

   The correct command to send custom metric data is put-metric-data under aws cloudwatch.

2. Step 2: Check the syntax and parameters

   The command requires --metric-name, --namespace, and --value parameters as shown in aws cloudwatch put-metric-data --metric-name CPUUtilization --namespace AWS/EC2 --value 70.

3. Final Answer:

   aws cloudwatch put-metric-data --metric-name CPUUtilization --namespace AWS/EC2 --value 70 -> Option D

4. Quick Check:

   Use put-metric-data with metric-name and namespace [OK]

Hint: Use 'put-metric-data' with metric-name and namespace [OK]

Common Mistakes:

• Using incorrect command names like send-metric or create-metric
• Omitting the namespace parameter
• Confusing metric creation with data sending

3. Given the following AWS CloudWatch custom metric data sent every minute:

aws cloudwatch put-metric-data --metric-name PageViews --namespace MyApp --value 5

What will be the average value shown in CloudWatch after 3 minutes if the command is run 3 times with values 5, 7, and 8 respectively?

medium

A. 6.67

B. 7

C. 20

D. 5

Solution

1. Step 1: Sum the metric values sent

   The values sent are 5, 7, and 8. Their sum is 5 + 7 + 8 = 20.

2. Step 2: Calculate the average over 3 data points

   Average = Total sum / Number of data points = 20 / 3 ≈ 6.67.

3. Final Answer:

   6.67 -> Option A

4. Quick Check:

   Average = sum/3 = 6.67 [OK]

Hint: Average = sum of values divided by count [OK]

Common Mistakes:

• Adding values but not dividing by count
• Taking the last value as average
• Multiplying values instead of averaging

4. You tried to send a custom metric to CloudWatch but it does not appear in the console. Which of the following is the most likely cause?

medium

A. CloudWatch only shows metrics after 24 hours delay.

B. You forgot to specify the metric namespace in the command.

C. Custom metrics are automatically visible without sending data.

D. Default metrics do not require namespaces, so this is normal.

Solution

1. Step 1: Check required parameters for custom metrics

   Custom metrics require a namespace to organize and display them properly in CloudWatch.

2. Step 2: Understand impact of missing namespace

   If the namespace is missing, CloudWatch cannot categorize the metric, so it won't appear in the console.

3. Final Answer:

   You forgot to specify the metric namespace in the command. -> Option B

4. Quick Check:

   Missing namespace hides custom metric [OK]

Hint: Always specify namespace when sending custom metrics [OK]

Common Mistakes:

• Assuming default metrics need namespaces
• Expecting immediate visibility without data sent
• Believing CloudWatch delays metric display by 24 hours

5. You want to monitor a custom application metric that tracks the number of user logins per hour. The default AWS metrics do not provide this data. What is the best approach to achieve this?

hard

A. Enable detailed monitoring on EC2 instances to get user login data.

B. Use the default CPUUtilization metric to estimate user logins.

C. Send custom metric data for user logins to CloudWatch using a unique namespace and metric name.

D. Create an alarm on default network metrics to infer user logins.

Solution

1. Step 1: Identify the need for custom data

   User logins per hour is specific to the application and not tracked by default AWS metrics.

2. Step 2: Use custom metrics for application-specific data

   Sending custom metric data with a unique namespace and metric name allows tracking exactly this information in CloudWatch.

3. Final Answer:

   Send custom metric data for user logins to CloudWatch using a unique namespace and metric name. -> Option C

4. Quick Check:

   Custom metrics track app-specific data [OK]

Hint: Use custom metrics for app-specific tracking [OK]

Common Mistakes:

• Trying to use unrelated default metrics for app data
• Assuming detailed monitoring tracks application events
• Using alarms on unrelated metrics to guess user activity