Data access logs in GCP - Step-by-Step Execution

Process Flow - Data access logs

Enable Data Access Logs

↓

Log Generated on Data Access

↓

Log Stored in Logging Service

↓

User Views Logs for Audit or Debug

↓

Use Logs to Monitor or Investigate

Data access logs start when logging is enabled, then logs are created on data access events, stored securely, and finally reviewed for auditing or troubleshooting.

Execution Sample

GCP

gcloud logging sinks create my-sink storage.googleapis.com/my-bucket --log-filter="resource.type=bigquery_resource AND protoPayload.methodName=\"jobservice.jobcompleted\""

This command creates a sink to export BigQuery data access logs to a Cloud Storage bucket.

Process Table

Step	Action	Input/Condition	Result/Output	Next Step
1	Enable data access logs	Set audit config in IAM or Admin Console	Data access logs start recording	2
2	Access data resource	User runs query or reads data	Log entry created for access event	3
3	Log entry stored	Log entry sent to Cloud Logging	Log stored securely and indexed	4
4	Create sink for logs	Define sink with filter and destination	Logs exported to chosen storage	5
5	View logs	User queries logs in Cloud Console or CLI	Logs displayed for audit or debug	6
6	Analyze logs	User reviews logs for anomalies or usage	Insights gained for security or billing	End

💡 Logs are continuously generated and stored; process ends when user finishes analysis.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 5	Final
Data Access Logs Enabled	False	True	True	True	True	True
Log Entry Created	None	Created	Stored	Stored	Exported	Viewed
Sink Configuration	None	None	None	Created	Created	Created
User Log View	None	None	None	None	Viewed	Viewed

Key Moments - 3 Insights

Why do logs only appear after enabling data access logs?

Because as shown in execution_table step 1, logs start recording only after enabling the audit config; before that, no logs are generated.

What happens if no sink is created for logs?

Logs remain in Cloud Logging but are not exported; step 4 shows sink creation is needed to export logs to storage or other destinations.

Can logs be viewed before they are stored?

No, logs must be stored first (step 3) before they can be viewed or exported (steps 5 and 6).

Visual Quiz - 3 Questions

Look at the execution_table, at which step are logs exported to storage?

A Step 5

B Step 3

C Step 4

D Step 6

💡 Hint

Check the 'Result/Output' column for step 4 about logs exported to chosen storage.

According to variable_tracker, when does 'Log Entry Created' change from None to Created?

A After Step 1

B After Step 2

C After Step 3

D After Step 4

💡 Hint

Look at the 'Log Entry Created' row and see when it first changes from None.

If data access logs are not enabled, what will be the state of 'Data Access Logs Enabled' in variable_tracker?

A False

B True

C Created

D Viewed

💡 Hint

Refer to the 'Data Access Logs Enabled' row in variable_tracker at Start.

Concept Snapshot

Data Access Logs in GCP:
- Enable audit logging for data access in IAM or Admin Console.
- Logs record every data read or query event.
- Logs are stored in Cloud Logging automatically.
- Create sinks to export logs to storage or BigQuery.
- View and analyze logs for auditing and troubleshooting.

Full Transcript

Data access logs in Google Cloud Platform start when you enable audit logging for data access. Once enabled, every time someone reads or queries data, a log entry is created. These logs are stored securely in Cloud Logging. You can create sinks to export these logs to other storage like Cloud Storage or BigQuery for long-term retention or analysis. Finally, you can view and analyze these logs in the Cloud Console or via command line to monitor usage, audit access, or troubleshoot issues. The process flow begins with enabling logs, continues with log creation on data access, storing logs, exporting via sinks, and ends with user viewing and analyzing logs.

Practice

1. What is the main purpose of data access logs in Google Cloud?

easy

A. To reduce storage costs

B. To record who accessed or changed your cloud data

C. To automatically backup your data

D. To increase the speed of data processing

Solution

1. Step 1: Understand what data access logs record

   Data access logs track user actions on cloud data, such as access or changes.

2. Step 2: Identify the main purpose

   The main goal is to monitor and audit who accessed or modified data for security and compliance.

3. Final Answer:

   To record who accessed or changed your cloud data -> Option B

4. Quick Check:

   Data access logs = record access and changes [OK]

Hint: Logs track data access and changes, not performance or cost [OK]

Common Mistakes:

• Confusing logs with backup or performance tools
• Thinking logs reduce costs automatically
• Assuming logs speed up data processing

2. Which of the following is the correct way to enable data access logs for a Google Cloud service?

easy

A. Install a logging agent on your local machine

B. Create a firewall rule to allow logging

C. Use Cloud Storage lifecycle rules

D. Enable audit logs in Cloud IAM & Admin for the service

Solution

1. Step 1: Identify how to enable data access logs

   Data access logs are enabled by configuring audit logs in Cloud IAM & Admin for the specific service.

2. Step 2: Eliminate unrelated options

   Firewall rules, local agents, and storage lifecycle rules do not enable audit logging.

3. Final Answer:

   Enable audit logs in Cloud IAM & Admin for the service -> Option D

4. Quick Check:

   Audit logs enabled via IAM & Admin [OK]

Hint: Audit logs are enabled in IAM & Admin, not by firewall or agents [OK]

Common Mistakes:

• Confusing firewall rules with logging setup
• Thinking local agents enable cloud audit logs
• Using storage lifecycle rules for logging

3. Given this filter for a logging sink:
resource.type="bigquery_resource" AND protoPayload.methodName="tableservice.insert"
What logs will this sink collect?

medium

A. All Cloud Storage bucket creations

B. All BigQuery query executions

C. All BigQuery table creation events

D. All Compute Engine VM starts

Solution

1. Step 1: Analyze the filter components

   The filter targets resource type 'bigquery_resource' and method 'tableservice.insert', which means table creation in BigQuery.

2. Step 2: Match filter to log events

   This filter collects logs about BigQuery tables being created, not queries or other services.

3. Final Answer:

   All BigQuery table creation events -> Option C

4. Quick Check:

   Filter matches BigQuery table inserts [OK]

Hint: Look for resource type and method name to identify event type [OK]

Common Mistakes:

• Confusing table creation with query execution
• Mixing BigQuery with Cloud Storage or Compute Engine
• Ignoring methodName in filter

4. You created a logging sink to export data access logs to BigQuery but see no data arriving. What is a likely cause?

medium

A. Data access logs are disabled in the project

B. BigQuery dataset is in a different region

C. The sink uses a Cloud Storage bucket instead of BigQuery

D. The sink filter excludes data access logs

Solution

1. Step 1: Check if data access logs are enabled

   If data access logs are disabled, no logs will be generated or exported.

2. Step 2: Consider other causes

   While filters and regions matter, the most common cause is logs not enabled at all.

3. Final Answer:

   Data access logs are disabled in the project -> Option A

4. Quick Check:

   Logs must be enabled to export data [OK]

Hint: Verify logs are enabled before checking sinks [OK]

Common Mistakes:

• Assuming filter or region is always the problem
• Confusing sink destination with log generation
• Not enabling data access logs in IAM & Admin

5. You want to audit all data access to Cloud Storage buckets in your project and store logs in BigQuery for analysis. Which steps should you take?

hard

A. Enable data access audit logs for Cloud Storage, create a sink with filter resource.type="gcs_bucket", export to BigQuery dataset

B. Enable system logs only, create a sink exporting all logs to Cloud Storage

C. Install logging agents on VMs, export logs to BigQuery without filters

D. Create a firewall rule to allow logging traffic, enable data access logs

Solution

1. Step 1: Enable data access audit logs for Cloud Storage

   This ensures logs capture who accessed or changed Cloud Storage buckets.

2. Step 2: Create a sink with filter resource.type="gcs_bucket" and export to BigQuery

   This filters logs to only Cloud Storage bucket access and stores them in BigQuery for analysis.

3. Final Answer:

   Enable data access audit logs for Cloud Storage, create a sink with filter resource.type="gcs_bucket", export to BigQuery dataset -> Option A

4. Quick Check:

   Enable logs + filter + export to BigQuery = audit setup [OK]

Hint: Enable logs, filter by resource, export to BigQuery [OK]

Common Mistakes:

• Using system logs instead of data access logs
• Exporting all logs without filtering
• Relying on firewall rules for logging