Audit logging in GCP - Deep Dive

Overview - Audit logging

What is it?

Audit logging is a way to record who did what, when, and where in a cloud system. It keeps track of actions taken on resources, like creating or deleting files, or changing settings. This helps organizations see activity history and find problems or security issues. Audit logs are like a diary for cloud services, showing detailed records of events.

Why it matters

Without audit logging, it would be very hard to know if someone made a mistake or tried to do something bad in your cloud system. It helps catch unauthorized actions and supports fixing problems quickly. For businesses, audit logs are important for security, compliance with laws, and understanding how systems are used. Without them, trust and safety in cloud environments would be much weaker.

Where it fits

Before learning audit logging, you should understand basic cloud concepts like resources, users, and permissions. After audit logging, you can explore security monitoring, incident response, and compliance management. Audit logging connects cloud operations with security and governance practices.

Mental Model

Core Idea

Audit logging is the cloud's detailed diary that records every important action to help track and understand what happened and who did it.

Think of it like...

Imagine a security camera in a store that records every customer’s actions. Audit logging is like that camera but for cloud systems, capturing every important move so you can review it later.

┌─────────────────────────────┐
│        Cloud System         │
├─────────────┬───────────────┤
│ User Action │ Resource Used │
├─────────────┼───────────────┤
│ Create VM   │ VM Instance   │
│ Delete File │ Storage Bucket│
│ Change IAM  │ Permissions   │
└─────┬───────┴───────┬───────┘
      │               │
      ▼               ▼
┌─────────────────────────────┐
│        Audit Logs           │
│ - Who did it                │
│ - What was done             │
│ - When it happened          │
│ - Where it happened         │
└─────────────────────────────┘

Build-Up - 6 Steps

1

Foundation

What is Audit Logging in Cloud

Concept: Audit logging records actions taken on cloud resources to keep a history.

In Google Cloud Platform (GCP), audit logs automatically record activities like creating, modifying, or deleting resources. These logs include details about who performed the action, what was done, and when. They help track changes and monitor usage.

Result

You get a continuous record of all important actions in your cloud environment.

Understanding that audit logs are automatic records helps you see how cloud platforms keep track of activity without extra setup.

2

Foundation

Types of Audit Logs in GCP

Concept: GCP provides different kinds of audit logs for various purposes.

There are three main types of audit logs in GCP: 1. Admin Activity Logs: Record changes to resources (like creating or deleting). 2. Data Access Logs: Record read or write access to data. 3. System Event Logs: Record system actions by Google services. Each type helps track different aspects of cloud usage.

Result

You can see detailed records for changes, data access, and system events separately.

Knowing the types of logs helps you focus on the right information for security or troubleshooting.

3

Intermediate

How Audit Logs Capture User Identity

🤔Before reading on: do you think audit logs always show the real user or sometimes a service acting on their behalf? Commit to your answer.

Concept: Audit logs record the identity of the user or service that performed an action.

Audit logs include the identity of the actor, which can be a human user or a service account (an automated system). This helps trace actions back to the source. Sometimes, actions happen through delegated permissions, and logs show the original user and the service acting for them.

Result

You can identify who or what caused each action, even if it was automated.

Understanding user identity in logs is key to tracing responsibility and detecting misuse.

4

Intermediate

Configuring Audit Log Sinks

🤔Before reading on: do you think audit logs are stored forever by default or need configuration to export and keep them? Commit to your answer.

Concept: Audit logs can be exported to other storage or analysis tools using sinks.

By default, audit logs are stored for a limited time in Cloud Logging. To keep logs longer or analyze them elsewhere, you create sinks that export logs to destinations like Cloud Storage, BigQuery, or Pub/Sub. This helps with long-term storage, compliance, or real-time alerting.

Result

You can save audit logs outside GCP or use them in custom monitoring systems.

Knowing how to export logs lets you build stronger security and compliance workflows.

5

Advanced

Interpreting Audit Log Entries

🤔Before reading on: do you think audit logs only show success events or also failures? Commit to your answer.

Concept: Audit logs include detailed event data, including success or failure of actions.

Each audit log entry contains fields like method name, resource name, timestamp, and status. Status shows if the action succeeded or failed. This helps detect unauthorized attempts or errors. Logs also include request and response metadata for deeper analysis.

Result

You can understand not just what happened, but if it worked or failed and why.

Recognizing failure events in logs is crucial for security incident detection and troubleshooting.

6

Expert

Audit Logging Impact on Performance and Costs

🤔Before reading on: do you think enabling all audit logs has no cost or performance impact? Commit to your answer.

Concept: Audit logging can affect system performance and incur costs depending on volume and retention.

Enabling detailed audit logs, especially Data Access logs, can generate large volumes of data. This may increase storage costs and affect system performance due to logging overhead. Experts balance logging detail with cost and performance by selectively enabling logs and using sinks efficiently.

Result

You manage audit logging to optimize security visibility without unnecessary expense or slowdowns.

Understanding the tradeoff between logging detail and resource use helps design scalable, cost-effective cloud security.

Under the Hood

Audit logging in GCP works by intercepting API calls made to cloud services. When a user or service requests an action, the system records metadata about the request, including identity, timestamp, resource, and outcome. These records are stored in Cloud Logging, where they can be queried or exported. The logging system is integrated deeply into the cloud platform's control plane, ensuring near real-time capture of events.

Why designed this way?

Audit logging was designed to be automatic and integrated to avoid relying on users to remember to log actions. This ensures comprehensive coverage and reduces blind spots. The separation into log types allows balancing detail and cost. Export sinks provide flexibility for compliance and analysis. Alternatives like manual logging were rejected because they are error-prone and incomplete.

┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│ User/Service  │─────▶│ Cloud API Call │─────▶│ Audit Logging │
│   Action      │      │  Interceptor   │      │  System      │
└───────────────┘      └───────────────┘      └──────┬────────┘
                                                      │
                                                      ▼
                                             ┌─────────────────┐
                                             │ Cloud Logging    │
                                             │ Storage & Query  │
                                             └─────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: do you think audit logs record every single action by default? Commit to yes or no.

Common Belief: Audit logs capture every action automatically without any configuration.

Reality: Only Admin Activity logs are enabled by default; Data Access logs must be explicitly enabled due to volume and cost.

Why it matters: Assuming all logs are recorded can lead to missing critical data during investigations.

Quick: do you think audit logs can be altered or deleted by users? Commit to yes or no.

Common Belief: Users can modify or delete audit logs like normal files.

Reality: Audit logs are protected and cannot be altered or deleted by regular users to ensure integrity and trustworthiness.

Why it matters: Believing logs can be changed undermines trust in audit data and can lead to poor security decisions.

Quick: do you think audit logs only record successful actions? Commit to yes or no.

Common Belief: Audit logs only show actions that succeeded, ignoring failures.

Reality: Audit logs record both successful and failed attempts, which is important for detecting unauthorized access or errors.

Why it matters: Ignoring failure logs can cause missed security breaches or operational issues.

Quick: do you think audit logging slows down cloud services significantly? Commit to yes or no.

Common Belief: Audit logging always causes noticeable performance problems.

Reality: While audit logging adds some overhead, it is optimized to minimize impact; excessive logging can affect performance but is controllable.

Why it matters: Overestimating impact may cause teams to disable important logs, reducing security visibility.

Expert Zone

1

Audit logs include 'principalSubject' fields that show the real user behind service accounts, which is often overlooked.

2

Log entries can be correlated with Cloud IAM policies to understand if an action was authorized or not.

3

Exported audit logs can be integrated with SIEM (Security Information and Event Management) tools for advanced threat detection.

When NOT to use

Audit logging is not a replacement for real-time intrusion detection or application-level logging. For detailed application behavior, use application logs. For real-time alerts, combine audit logs with monitoring tools. Avoid enabling all Data Access logs in high-volume environments without filtering due to cost and performance.

Production Patterns

In production, teams enable Admin Activity logs by default and selectively enable Data Access logs for sensitive resources. They export logs to BigQuery for analysis and set up alerts on suspicious activities. Logs are retained according to compliance needs, often integrated with centralized security dashboards.

Connections

Security Information and Event Management (SIEM)

Audit logs feed into SIEM systems for centralized security monitoring and alerting.

Understanding audit logging helps grasp how raw event data becomes actionable security intelligence in SIEM.

Version Control Systems

Both audit logs and version control track changes over time with author and timestamp metadata.

Knowing audit logging clarifies how change tracking in software development parallels tracking actions in cloud systems.

Forensic Science

Audit logging is like digital forensics, collecting evidence to reconstruct events after an incident.

Recognizing audit logs as forensic evidence highlights their role in investigation and accountability.

Common Pitfalls

#1 Assuming all audit logs are enabled by default and missing critical data.

Wrong approach:

Not enabling Data Access logs and relying only on default Admin Activity logs.

Correct approach:

Explicitly enable Data Access logs for sensitive resources where detailed tracking is needed.

Root cause:

Misunderstanding default logging settings and the need for explicit configuration.

#2 Storing audit logs only in Cloud Logging without exporting for long-term retention.

Wrong approach:

Relying on default log retention (30 days) without setting up sinks.

Correct approach:

Create sinks to export logs to Cloud Storage or BigQuery for longer retention and analysis.

Root cause:

Unawareness of retention limits and export capabilities.

#3 Ignoring failed actions in audit logs and focusing only on successful events.

Wrong approach:

Filtering logs to show only success status entries.

Correct approach:

Include both success and failure events in log analysis to detect unauthorized attempts.

Root cause:

Misunderstanding the importance of failure events for security.

Key Takeaways

Audit logging is an automatic record of who did what and when in cloud systems, essential for security and troubleshooting.

GCP provides different types of audit logs, each capturing specific kinds of activity for better focus and cost control.

Audit logs include detailed information about user identity, action success or failure, and resource affected, enabling deep analysis.

Exporting audit logs to external storage or analysis tools is crucial for long-term retention and compliance.

Balancing audit logging detail with performance and cost is key to effective cloud security management.

Practice

1. What is the main purpose of audit logging in Google Cloud Platform?

easy

A. To create virtual machines

B. To record actions for security and troubleshooting

C. To speed up network traffic

D. To increase storage capacity automatically

Solution

1. Step 1: Understand audit logging function

   Audit logging captures user and system actions in your cloud environment.

2. Step 2: Identify the purpose

   This helps with security monitoring and troubleshooting issues.

3. Final Answer:

   To record actions for security and troubleshooting -> Option B

4. Quick Check:

   Audit logging = record actions [OK]

Hint: Audit logs track actions for security and fixes [OK]

Common Mistakes:

• Confusing audit logs with storage or network features
• Thinking audit logs create resources
• Assuming audit logs improve performance

2. Which command correctly creates a sink to export audit logs to a Cloud Storage bucket named my-logs-bucket?

easy

A. gcloud logging sinks create my-sink storage.googleapis.com/my-logs-bucket --log-filter="resource.type=\"gce_instance\""

B. gcloud logging create sink my-sink storage://my-logs-bucket

C. gcloud logs create sink my-sink gs://my-logs-bucket

D. gcloud logging sinks add my-sink storage.googleapis.com/my-logs-bucket

Solution

1. Step 1: Identify correct gcloud command syntax

   The correct command uses gcloud logging sinks create followed by sink name and destination URL.

2. Step 2: Verify destination format

   Cloud Storage buckets use storage.googleapis.com/BUCKET_NAME format.

3. Final Answer:

   gcloud logging sinks create my-sink storage.googleapis.com/my-logs-bucket --log-filter="resource.type=\"gce_instance\"" -> Option A

4. Quick Check:

   Correct sink creation syntax = gcloud logging sinks create my-sink storage.googleapis.com/my-logs-bucket --log-filter="resource.type=\"gce_instance\"" [OK]

Hint: Use 'gcloud logging sinks create' with full storage URL [OK]

Common Mistakes:

• Using wrong command verbs like 'create sink' or 'add sink'
• Incorrect bucket URL format (missing storage.googleapis.com)
• Using 'gs://' instead of full URL

3. Given this filter in a sink configuration: resource.type="bigquery_resource" AND protoPayload.methodName="jobservice.jobcompleted", what logs will be exported?

medium

A. Only logs from Compute Engine instances

B. All error logs from Cloud Storage

C. All BigQuery job completion audit logs

D. All network traffic logs

Solution

1. Step 1: Analyze the resource type filter

   The filter specifies resource.type="bigquery_resource", so logs relate to BigQuery.

2. Step 2: Analyze the method name filter

   protoPayload.methodName="jobservice.jobcompleted" means only job completion events are included.

3. Final Answer:

   All BigQuery job completion audit logs -> Option C

4. Quick Check:

   Filter matches BigQuery job completions = All BigQuery job completion audit logs [OK]

Hint: Filter resource and method to find exported logs [OK]

Common Mistakes:

• Confusing resource types with unrelated services
• Ignoring methodName filter
• Assuming all logs are exported

4. You created a sink to export audit logs to a Pub/Sub topic but logs are not appearing. What is the most likely cause?

medium

A. The sink lacks permission to publish to the Pub/Sub topic

B. The sink filter is too broad and excludes all logs

C. Audit logging is disabled in the project

D. The Pub/Sub topic name is misspelled in the sink

Solution

1. Step 1: Check sink permissions

   Sinks need permission to write to destinations like Pub/Sub topics.

2. Step 2: Identify common permission issue

   If logs do not appear, the sink likely lacks the pubsub.publisher role on the topic.

3. Final Answer:

   The sink lacks permission to publish to the Pub/Sub topic -> Option A

4. Quick Check:

   Missing sink permissions = The sink lacks permission to publish to the Pub/Sub topic [OK]

Hint: Check sink destination permissions first when logs missing [OK]

Common Mistakes:

• Assuming audit logging is off without checking
• Blaming filter without verifying permissions
• Ignoring typo in topic name

5. You want to export only admin activity audit logs for Cloud Storage buckets in your project to a BigQuery dataset for analysis. Which sink filter is correct?

hard

A. resource.type="gcs_bucket" AND severity="ERROR"

B. logName:"cloudaudit.googleapis.com/activity" AND resource.type="gcs_bucket" AND protoPayload.methodName:"storage.objects.create"

C. logName:"cloudaudit.googleapis.com/data_access" AND resource.type="gcs_bucket"

D. logName:"cloudaudit.googleapis.com/admin_activity" AND resource.type="gcs_bucket"

Solution

1. Step 1: Identify admin activity logs

   Admin activity logs use logName:"cloudaudit.googleapis.com/admin_activity".

2. Step 2: Filter for Cloud Storage buckets

   Use resource.type="gcs_bucket" to target storage buckets.

3. Step 3: Confirm filter matches requirement

   logName:"cloudaudit.googleapis.com/admin_activity" AND resource.type="gcs_bucket" correctly combines admin activity logs with GCS buckets.

4. Final Answer:

   logName:"cloudaudit.googleapis.com/admin_activity" AND resource.type="gcs_bucket" -> Option D

5. Quick Check:

   Admin activity + GCS bucket filter = logName:"cloudaudit.googleapis.com/admin_activity" AND resource.type="gcs_bucket" [OK]

Hint: Use admin_activity logName for admin logs [OK]

Common Mistakes:

• Using data_access logs instead of admin_activity
• Filtering by specific methods unnecessarily
• Confusing severity with log type