Data pipeline patterns in GCP - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a Cloud Storage bucket for raw data ingestion.

GCP

resource "google_storage_bucket" "raw_data_bucket" {
  name     = "my-raw-data-[1]"
  location = "US"
}

Drag options to blanks, or click blank then click option'

Abucket

Bstorage

Cpipeline

Ddata

Attempts:

3 left

💡 Hint

Common Mistakes

Using generic names like 'bucket' or 'storage' that don't describe the data.

Leaving the bucket name incomplete or too vague.

The bucket name should include a descriptive term like 'data' to indicate its purpose for raw data ingestion.

2fill in blank

medium

Complete the code to define a Pub/Sub topic for streaming data ingestion.

GCP

resource "google_pubsub_topic" "stream_topic" {
  name = "[1]-stream-topic"
}

Drag options to blanks, or click blank then click option'

Abatch

Bdata

Cstreaming

Draw

Attempts:

3 left

💡 Hint

Common Mistakes

Using 'batch' which is for batch processing, not streaming.

Using vague names that don't indicate the topic's purpose.

The topic name should clearly indicate it is related to data streaming, so 'data' is a good base name.

3fill in blank

hard

Fix the error in the Dataflow job configuration to specify the correct runner.

GCP

resource "google_dataflow_job" "etl_job" {
  name     = "etl-job"
  template_gcs_path = "gs://dataflow-templates/latest/Word_Count"
  [1] = "DataflowRunner"
  region   = "us-central1"
}

Drag options to blanks, or click blank then click option'

Atemplate_type

Bjob_type

Crunner

Dexecution_mode

Attempts:

3 left

💡 Hint

Common Mistakes

Using incorrect property names like 'job_type' or 'execution_mode'.

Omitting the runner property entirely.

The correct property to specify the runner in a Dataflow job is 'runner'.

4fill in blank

hard

Fill both blanks to create a BigQuery dataset with the correct location and description.

GCP

resource "google_bigquery_dataset" "analytics_dataset" {
  dataset_id = "analytics"
  location   = "[1]"
  description = "[2]"
}

Drag options to blanks, or click blank then click option'

AUS

BEU

CAnalytics data storage

DRaw data storage

Attempts:

3 left

💡 Hint

Common Mistakes

Choosing the wrong location like 'EU' when the data is US-based.

Using a vague or incorrect description.

The dataset location should be 'US' for this example, and the description should clearly state its purpose as 'Analytics data storage'.

5fill in blank

hard

Fill all three blanks to define a Cloud Composer environment with the correct name, location, and machine type.

GCP

resource "google_composer_environment" "data_pipeline_env" {
  name   = "[1]"
  region = "[2]"
  config {
    node_config {
      machine_type = "[3]"
    }
  }
}

Drag options to blanks, or click blank then click option'

Adata-pipeline-env

Bus-central1

Cn1-standard-1

Deurope-west1

Attempts:

3 left

💡 Hint

Common Mistakes

Using a region that doesn't match the environment's location.

Choosing an unsupported machine type.

The environment name should be descriptive like 'data-pipeline-env', the region 'us-central1' is a common GCP region, and 'n1-standard-1' is a typical machine type for Composer nodes.