Data Fusion for ETL in GCP - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ service_behavior

intermediate

How does Data Fusion handle schema changes during ETL?

In Google Cloud Data Fusion, when an ETL pipeline reads from a source and the source schema changes (e.g., a new column is added), what is the default behavior of the pipeline if no schema update is applied?

A The pipeline fails with a schema mismatch error and stops processing.

B The pipeline automatically updates the schema and processes the new column without errors.

C The pipeline ignores the new column and continues processing with the old schema.

D The pipeline duplicates the data for the new column causing data inconsistency.

💡 Hint

Think about how strict the schema enforcement is by default in Data Fusion pipelines.

By default, Data Fusion pipelines use the schema defined at design time. If the source schema changes, the pipeline ignores new columns unless the schema is updated manually or through schema registry integration.

❓ Architecture

intermediate

Choosing the right Data Fusion edition for complex ETL

You need to build an ETL pipeline in Google Cloud Data Fusion that requires custom plugins and advanced transformations. Which Data Fusion edition should you choose?

A Basic edition, because it supports all plugin types and transformations.

B Standard edition, because it supports only built-in plugins.

C Trial edition, because it has no limits on plugin usage.

D Enterprise edition, because it supports custom plugins and advanced transformations.

💡 Hint

Consider which edition supports extensibility and advanced features.

The Enterprise edition of Data Fusion supports custom plugins and advanced transformations, making it suitable for complex ETL scenarios.

❓ security

advanced

Securing sensitive data in Data Fusion pipelines

You have an ETL pipeline in Data Fusion that processes sensitive customer data. Which approach best protects this data during pipeline execution?

A Use Cloud KMS to encrypt data at rest and configure Data Fusion to use service accounts with least privilege.

B Store sensitive data in plain text and rely on network security only.

C Disable audit logs to prevent data exposure in logs.

D Use public service accounts with broad permissions for easy access.

💡 Hint

Think about encryption and access control best practices.

Encrypting data at rest with Cloud KMS and using least privilege service accounts ensures sensitive data is protected during processing and storage.

❓ Configuration

advanced

Configuring a Data Fusion pipeline to read from a Cloud Storage bucket

Which configuration correctly sets up a source plugin in Data Fusion to read CSV files from a Google Cloud Storage bucket named my-data-bucket?

A Set the source plugin type to 'BigQuery' and the table to 'my-data-bucket'.

B Set the source plugin type to 'GCS File' and the path to 'gs://my-data-bucket/*.csv'.

C Set the source plugin type to 'Database' and the connection string to 'gs://my-data-bucket'.

D Set the source plugin type to 'Kafka' and the topic to 'my-data-bucket'.

💡 Hint

Consider which plugin reads files from Cloud Storage.

The 'GCS File' source plugin reads files from Cloud Storage buckets using the 'gs://' URI format.

✅ Best Practice

expert

Optimizing Data Fusion pipelines for large-scale ETL workloads

Which practice best improves performance and scalability of a Data Fusion ETL pipeline processing terabytes of data?

A Use batch pipelines with partitioned sources and sinks, and enable pipeline parallelism.

B Use single-threaded pipelines to avoid concurrency issues.

C Store all intermediate data in local temporary files on the Data Fusion instance.

D Disable pipeline retries to reduce execution time.

💡 Hint

Think about how to handle large data volumes efficiently.

Partitioning data sources and sinks and enabling parallelism allows Data Fusion pipelines to process large data sets efficiently and scale horizontally.