Why observability matters in GCP - See It in Action

Why observability matters

📖 Scenario: You are managing a cloud application on Google Cloud Platform (GCP). You want to understand how your application behaves and quickly find problems when they happen.Observability helps you see inside your system like a doctor uses a stethoscope to listen to a patient’s heartbeat. It shows you logs, metrics, and traces so you can keep your app healthy.

🎯 Goal: Build a simple GCP monitoring setup that collects logs and metrics from a virtual machine instance and creates an alert policy to notify you if CPU usage is too high.

📋 What You'll Learn

Create a GCP Compute Engine VM instance with logging enabled

Set up a metric to monitor CPU utilization

Create an alert policy for CPU usage above 80%

Verify the alert policy is linked to the VM instance

💡 Why This Matters

🌍 Real World

Cloud engineers use observability to keep applications running smoothly by detecting issues early and understanding system behavior.

💼 Career

Knowing how to set up monitoring and alerting on GCP is essential for roles like Cloud Engineer, Site Reliability Engineer, and DevOps Engineer.

1

Create a GCP Compute Engine VM instance with logging enabled

Write a Terraform resource block named google_compute_instance called vm_instance that creates a VM instance with the name observability-vm in the zone us-central1-a. Enable the metadata enable-oslogin with value true and set the machine type to e2-medium. Also, enable the logging agent by adding the metadata google-logging-enabled with value true.

GCP

# Your code here

Hint

Use resource "google_compute_instance" "vm_instance" and set metadata keys for logging and OS login.

2

Set up a metric to monitor CPU utilization

Create a Terraform resource block named google_monitoring_metric_descriptor called cpu_utilization_metric that defines a custom metric with the type custom.googleapis.com/cpu/utilization. Set the metric kind to GAUGE and the value type to DOUBLE. Add a description "CPU utilization metric for VM".

GCP

resource "google_compute_instance" "vm_instance" {
  name         = "observability-vm"
  machine_type = "e2-medium"
  zone         = "us-central1-a"

  metadata = {
    "enable-oslogin"        = "true"
    "google-logging-enabled" = "true"
  }

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network = "default"
    access_config {}
  }
}

# Define a custom metric for CPU utilization
# Your code here

Hint

Use google_monitoring_metric_descriptor resource with the specified type and properties.

3

Create an alert policy for CPU usage above 80%

Add a Terraform resource block named google_monitoring_alert_policy called cpu_alert_policy. Set the display name to "High CPU Usage Alert". Add a condition with the display name "CPU usage above 80%" that triggers when the metric custom.googleapis.com/cpu/utilization is above 0.8 for at least 5 minutes. Use the combiner OR.

GCP

resource "google_compute_instance" "vm_instance" {
  name         = "observability-vm"
  machine_type = "e2-medium"
  zone         = "us-central1-a"

  metadata = {
    "enable-oslogin"        = "true"
    "google-logging-enabled" = "true"
  }

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network = "default"
    access_config {}
  }
}

resource "google_monitoring_metric_descriptor" "cpu_utilization_metric" {
  type        = "custom.googleapis.com/cpu/utilization"
  metric_kind = "GAUGE"
  value_type  = "DOUBLE"
  description = "CPU utilization metric for VM"
}

# Create an alert policy for CPU usage above 80%
# Your code here

Hint

Use google_monitoring_alert_policy with a condition_threshold for the custom CPU metric.

4

Link the alert policy to the VM instance

Add a notification_channels attribute to the google_monitoring_alert_policy resource called cpu_alert_policy. Use a placeholder string "projects/my-project/notificationChannels/1234567890" to simulate a notification channel ID.

GCP

resource "google_compute_instance" "vm_instance" {
  name         = "observability-vm"
  machine_type = "e2-medium"
  zone         = "us-central1-a"

  metadata = {
    "enable-oslogin"        = "true"
    "google-logging-enabled" = "true"
  }

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network = "default"
    access_config {}
  }
}

resource "google_monitoring_metric_descriptor" "cpu_utilization_metric" {
  type        = "custom.googleapis.com/cpu/utilization"
  metric_kind = "GAUGE"
  value_type  = "DOUBLE"
  description = "CPU utilization metric for VM"
}

resource "google_monitoring_alert_policy" "cpu_alert_policy" {
  display_name = "High CPU Usage Alert"
  combiner    = "OR"

  conditions {
    display_name = "CPU usage above 80%"

    condition_threshold {
      filter     = "metric.type = \"custom.googleapis.com/cpu/utilization\""
      comparison = "COMPARISON_GT"
      threshold_value = 0.8
      duration   = "300s"
    }
  }

  # Add notification channel here
  # Your code here
}

Hint

Add notification_channels attribute with the placeholder notification channel ID inside the alert policy resource.

Practice

1. Why is observability important in cloud systems?

easy

A. It helps you understand what is happening inside your system.

B. It automatically fixes all system errors without human help.

C. It replaces the need for backups and security.

D. It slows down the system to prevent crashes.

Solution

1. Step 1: Understand the purpose of observability

   Observability is about seeing inside your system to understand its behavior.

2. Step 2: Identify the correct benefit

   Among the options, only understanding system behavior matches observability's goal.

3. Final Answer:

   It helps you understand what is happening inside your system. -> Option A

4. Quick Check:

   Observability = Understanding system behavior [OK]

Hint: Observability means seeing inside your system clearly [OK]

Common Mistakes:

• Thinking observability fixes errors automatically
• Confusing observability with backups or security
• Believing observability slows down systems

2. Which of the following is a correct way to collect logs in Google Cloud for observability?

easy

A. Disable logging to improve performance.

B. Manually download logs from each server every hour.

C. Use Cloud Logging to gather and store logs from your services.

D. Send logs only to local files without cloud integration.

Solution

1. Step 1: Identify Google Cloud's logging tool

   Cloud Logging is the managed service for collecting and storing logs.

2. Step 2: Evaluate options for best practice

   Manual downloads and local-only logs are inefficient; disabling logs harms observability.

3. Final Answer:

   Use Cloud Logging to gather and store logs from your services. -> Option C

4. Quick Check:

   Cloud Logging = Correct log collection method [OK]

Hint: Use Cloud Logging for automatic, centralized log collection [OK]

Common Mistakes:

• Thinking manual log downloads are practical
• Disabling logs to improve speed
• Ignoring cloud-based log storage

3. Given this snippet using Google Cloud Monitoring API:

metrics = client.list_time_series(
  name="projects/my-project",
  filter="metric.type=\"compute.googleapis.com/instance/cpu/utilization\"",
  interval=interval,
  view=client.TimeSeriesView.FULL
)
print(len(metrics))

What does len(metrics) represent?

medium

A. The number of CPU cores in the project.

B. The count of time series matching the CPU utilization metric filter.

C. The total CPU usage percentage across all instances.

D. The number of projects in Google Cloud.

Solution

1. Step 1: Understand the API call

   The code lists time series data filtered by CPU utilization metric for a project.

2. Step 2: Interpret the length of the result

   Length of metrics is the count of time series matching the filter, not CPU cores or usage sums.

3. Final Answer:

   The count of time series matching the CPU utilization metric filter. -> Option B

4. Quick Check:

   len(metrics) = Number of matching time series [OK]

Hint: len() on list_time_series = count of matching time series [OK]

Common Mistakes:

• Confusing time series count with CPU core count
• Thinking len() returns CPU usage percentage
• Assuming it counts projects instead of metrics

4. You set up Cloud Trace but notice no traces appear in the console. What is a likely cause?

medium

A. Cloud Trace automatically collects all traces without setup.

B. Cloud Trace requires disabling logging to work.

C. Traces are only visible after 24 hours delay.

D. The application is not sending trace data to Cloud Trace.

Solution

1. Step 1: Check trace data flow

   Cloud Trace needs the application to send trace data explicitly.

2. Step 2: Evaluate other options

   Trace data is not automatic, no 24-hour delay, and logging is unrelated to trace visibility.

3. Final Answer:

   The application is not sending trace data to Cloud Trace. -> Option D

4. Quick Check:

   Missing trace data = app not sending traces [OK]

Hint: Traces appear only if app sends them to Cloud Trace [OK]

Common Mistakes:

• Assuming traces appear automatically
• Thinking traces have long delays
• Disabling logging to fix trace issues

5. You want to improve your system's observability by combining logs, metrics, and traces in Google Cloud. Which approach best achieves this?

hard

A. Use Cloud Logging for logs, Cloud Monitoring for metrics, and Cloud Trace for traces, then analyze them together.

B. Store logs only in local files, ignore metrics, and use Cloud Trace alone.

C. Disable metrics collection to reduce cost and rely on logs only.

D. Send all data to Cloud Storage and analyze manually without specialized tools.

Solution

1. Step 1: Identify Google Cloud observability tools

   Cloud Logging handles logs, Cloud Monitoring handles metrics, and Cloud Trace handles traces.

2. Step 2: Understand best practice for combined observability

   Using all three tools together provides a full picture and easier analysis.

3. Final Answer:

   Use Cloud Logging for logs, Cloud Monitoring for metrics, and Cloud Trace for traces, then analyze them together. -> Option A

4. Quick Check:

   Combine logs, metrics, traces with GCP tools = Use Cloud Logging for logs, Cloud Monitoring for metrics, and Cloud Trace for traces, then analyze them together. [OK]

Hint: Use all three GCP tools together for full observability [OK]

Common Mistakes:

• Ignoring metrics or traces
• Storing logs only locally
• Disabling metrics to save cost
• Manual analysis without tools