Default vs custom metrics in AWS - Trade-offs & Expert Analysis

Overview - Default vs custom metrics

What is it?

Metrics are measurements that tell you how well your cloud resources are working. Default metrics are automatically collected by cloud services without extra setup. Custom metrics are measurements you create yourself to track things specific to your needs. Both help you understand and improve your cloud systems.

Why it matters

Without metrics, you would be guessing if your cloud resources are healthy or efficient. Default metrics give basic health info, but they might miss important details unique to your app. Custom metrics fill that gap, helping you catch problems early and optimize performance. Without them, outages or slowdowns could go unnoticed until users complain.

Where it fits

Before learning this, you should understand basic cloud services and monitoring concepts. After this, you can learn how to set alarms and automate responses based on metrics. This knowledge fits into managing cloud infrastructure and ensuring reliable applications.

Mental Model

Core Idea

Metrics are like the vital signs of your cloud resources, with default metrics as the basic checkup and custom metrics as specialized tests you choose.

Think of it like...

Imagine going to a doctor: default metrics are like standard blood pressure and temperature checks done for everyone, while custom metrics are like specific allergy tests or scans tailored to your unique health needs.

┌───────────────┐       ┌───────────────┐
│ Default       │       │ Custom        │
│ Metrics      │       │ Metrics       │
│ (Auto-collected)│       │ (User-defined) │
└──────┬────────┘       └──────┬────────┘
       │                       │
       │                       │
       ▼                       ▼
  Basic health info       Specific insights
       │                       │
       └────────────┬──────────┘
                    ▼
             Cloud Monitoring

Build-Up - 7 Steps

1

Foundation

What are metrics in cloud monitoring

Concept: Introduce the basic idea of metrics as measurements of cloud resource behavior.

Metrics are numbers that tell you how your cloud resources like servers, databases, or applications are performing. For example, CPU usage or network traffic. They help you see if things are working well or if there are problems.

Result

You understand that metrics are essential data points for monitoring cloud systems.

Knowing what metrics are is the foundation for all cloud monitoring and management.

2

Foundation

Understanding default metrics

Concept: Explain default metrics as those automatically provided by cloud services.

Cloud providers like AWS automatically collect common metrics for their services. For example, EC2 instances report CPU utilization and disk reads without you doing anything. These are called default metrics because they come ready-made.

Result

You can identify default metrics and know they require no extra setup.

Recognizing default metrics helps you quickly get basic health info without extra work.

3

Intermediate

Why create custom metrics

🤔Before reading on: do you think default metrics cover all monitoring needs or only basic ones? Commit to your answer.

Concept: Introduce the need for custom metrics to track specific application or business data.

Default metrics cover general system health but might miss details important to your app. Custom metrics let you measure things like user logins, order counts, or error rates that default metrics don't track. You send these metrics to the cloud monitoring service yourself.

Result

You understand the purpose and value of custom metrics.

Knowing why custom metrics exist helps you design better monitoring tailored to your unique needs.

4

Intermediate

How to create and send custom metrics

🤔Before reading on: do you think custom metrics are automatically collected or require manual setup? Commit to your answer.

Concept: Explain the process of defining and sending custom metrics to the cloud monitoring system.

To create custom metrics, you write code or use tools to send your specific data points to the cloud monitoring service. For example, in AWS CloudWatch, you use the PutMetricData API to send your custom metric values regularly.

Result

You know the practical steps to implement custom metrics.

Understanding how to send custom metrics empowers you to monitor what matters most.

5

Intermediate

Comparing default and custom metrics features

🤔Before reading on: do you think default and custom metrics have the same cost and retention policies? Commit to your answer.

Concept: Highlight differences in cost, retention, and granularity between default and custom metrics.

Default metrics are usually free or low cost and retained for a standard period. Custom metrics often cost more and you decide how long to keep them. Also, custom metrics can have finer detail or track unique data, while default metrics are fixed.

Result

You can weigh tradeoffs when choosing metrics to monitor.

Knowing differences helps optimize monitoring costs and effectiveness.

6

Advanced

Using custom metrics for proactive alerting

🤔Before reading on: do you think alerts can be based only on default metrics or also on custom metrics? Commit to your answer.

Concept: Show how custom metrics enable precise alerts for business or app-specific conditions.

You can set alarms that trigger when custom metrics cross thresholds, like too many failed logins or slow response times. This helps catch issues before users notice. Default metrics alone might not detect these specific problems.

Result

You see how custom metrics improve operational awareness.

Understanding alerting with custom metrics enables proactive system management.

7

Expert

Challenges and best practices with custom metrics

🤔Before reading on: do you think sending many custom metrics can impact system performance or cost? Commit to your answer.

Concept: Discuss performance, cost, and design considerations when using custom metrics at scale.

Sending too many custom metrics can increase costs and add overhead to your systems. Best practices include batching metric data, choosing meaningful metrics, and cleaning up unused ones. Also, consider metric naming conventions and units for clarity.

Result

You learn how to use custom metrics efficiently and avoid pitfalls.

Knowing these challenges helps maintain scalable and cost-effective monitoring.

Under the Hood

Cloud monitoring systems collect metrics by gathering data points from resources at regular intervals. Default metrics come from built-in agents or service APIs that automatically report common stats. Custom metrics require your code or tools to send data via APIs to the monitoring backend, where it is stored, aggregated, and made queryable.

Why designed this way?

Default metrics were designed to provide quick, easy health checks without user effort. Custom metrics were added later to allow flexibility for diverse applications and business needs. This separation balances ease of use with customization, avoiding overload of irrelevant data.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Cloud Service │──────▶│ Default       │       │ Custom        │
│ Resources    │       │ Metrics       │       │ Metrics       │
│ (EC2, RDS)   │       │ (Auto-collected)│       │ (User-sent)   │
└──────┬────────┘       └──────┬────────┘       └──────┬────────┘
       │                       │                       │
       │                       │                       │
       ▼                       ▼                       ▼
  Metrics Data           Metrics Data           Metrics Data
  Collection             Storage & Query        Storage & Query
       │                       │                       │
       └───────────────┬───────┴───────────────┬───────┘
                       ▼                       ▼
                 Monitoring Dashboard & Alerts

Myth Busters - 4 Common Misconceptions

Quick: do you think default metrics cover all monitoring needs? Commit yes or no.

Common Belief: Default metrics are enough to monitor any cloud application fully.

Reality: Default metrics provide only basic system health data and often miss application-specific or business-related insights.

Why it matters: Relying only on default metrics can cause missed problems or slow reactions to issues unique to your app.

Quick: do you think custom metrics are free like default metrics? Commit yes or no.

Common Belief: Custom metrics cost the same as default metrics or are free.

Reality: Custom metrics usually incur additional costs based on volume and retention time, unlike many default metrics which are free or cheaper.

Why it matters: Ignoring cost differences can lead to unexpected bills and budget overruns.

Quick: do you think custom metrics are collected automatically? Commit yes or no.

Common Belief: Custom metrics are automatically collected by the cloud provider like default metrics.

Reality: Custom metrics must be explicitly sent by your code or tools; they are not collected automatically.

Why it matters: Assuming automatic collection can cause missing critical data and blind spots in monitoring.

Quick: do you think sending many custom metrics has no impact on system performance? Commit yes or no.

Common Belief: Sending many custom metrics does not affect system performance or cost significantly.

Reality: High volume of custom metrics can increase network load, processing overhead, and costs.

Why it matters: Not managing custom metrics volume can degrade system performance and inflate expenses.

Expert Zone

1

Custom metrics require careful naming conventions and units to ensure clarity and avoid confusion in dashboards and alerts.

2

Batching metric data points before sending reduces API calls and improves performance and cost efficiency.

3

Retention policies for custom metrics should balance historical insight needs with storage costs, often requiring lifecycle management.

When NOT to use

Avoid custom metrics when default metrics already provide sufficient insight or when the cost and complexity outweigh the benefits. Instead, use enhanced default metrics or logs for troubleshooting.

Production Patterns

In production, teams use custom metrics to track business KPIs like transaction rates or user engagement alongside default system metrics. They integrate these metrics with alerting and automated scaling to maintain reliability and performance.

Connections

Application Performance Monitoring (APM)

Builds-on

Understanding metrics is foundational to APM tools that combine metrics, traces, and logs for full visibility.

Financial Budgeting

Analogy

Just like tracking expenses helps manage budgets, monitoring metrics helps manage cloud resource costs and performance.

Human Vital Signs Monitoring

Similar pattern

Both track key indicators to detect early signs of problems and maintain health, whether in humans or cloud systems.

Common Pitfalls

#1 Assuming default metrics cover all monitoring needs.

Wrong approach:

Relying only on CPU and memory usage metrics to detect application errors.

Correct approach:

Implement custom metrics to track application-specific errors and business events.

Root cause:

Misunderstanding the scope and limitations of default metrics.

#2 Sending custom metrics too frequently without batching.

Wrong approach:

Calling PutMetricData API for every single event immediately.

Correct approach:

Batch multiple metric data points and send them together periodically.

Root cause:

Not considering API rate limits and performance impact.

#3 Ignoring cost implications of custom metrics.

Wrong approach:

Creating hundreds of custom metrics without monitoring usage or cost.

Correct approach:

Plan and monitor custom metric volume and retention to control costs.

Root cause:

Lack of awareness about pricing models for custom metrics.

Key Takeaways

Default metrics provide automatic, basic health data for cloud resources but may miss application-specific details.

Custom metrics let you track unique, important data tailored to your application's needs, improving monitoring precision.

Creating custom metrics requires manual setup and careful management to balance insight, performance, and cost.

Effective monitoring combines both default and custom metrics to maintain cloud system health and business success.

Understanding the differences and best practices around metrics empowers proactive, cost-effective cloud management.