Why sources define raw data contracts in dbt - Quick Recap

Recall & Review

beginner

What is a raw data contract in the context of data sources?

A raw data contract is an agreement or set of rules that defines the expected structure, format, and quality of raw data coming from a source before it is processed or transformed.

beginner

Why do data teams define raw data contracts for sources?

To ensure data consistency, reliability, and to catch errors early by setting clear expectations about the data's format and quality before it enters the transformation pipeline.

intermediate

How do raw data contracts help in data transformation workflows?

They act as a checkpoint that validates incoming data, preventing bad or unexpected data from causing errors downstream in transformations or analyses.

intermediate

What might happen if raw data contracts are not defined for sources?

Without raw data contracts, data inconsistencies or errors can go unnoticed, leading to incorrect analysis, broken pipelines, and loss of trust in data.

beginner

Give an example of a rule that might be included in a raw data contract.

An example rule could be: "The column 'user_id' must always be a non-null integer," ensuring that every record has a valid user identifier.

What is the main purpose of defining raw data contracts for sources?

A To create visualizations directly from raw data

B To speed up data transformation by skipping validation

C To store data in a compressed format

D To ensure data meets expected structure and quality before processing

Raw data contracts set expectations for data format and quality to catch issues early.

Which of the following is NOT a benefit of raw data contracts?

A Improved data reliability

B Early detection of data errors

C Automatic data visualization

D Clear communication of data expectations

Raw data contracts do not create visualizations; they focus on data quality and structure.

What could happen if raw data contracts are missing?

A Data will automatically be corrected

B Data pipelines may break due to unexpected data

C Data will be faster to process without checks

D Data will be encrypted

Without contracts, unexpected data can cause errors in pipelines.

A raw data contract might specify that a column must be:

A Non-null and of a specific data type

B Encrypted

C Randomly generated

D Always null

Contracts define expected data types and nullability to ensure data quality.

In dbt, why is defining raw data contracts important before transformations?

A To avoid errors and maintain trust in transformed data

B To skip testing transformed data

C To reduce storage costs

D To create dashboards automatically

Contracts help catch issues early, ensuring transformations work on clean data.

Explain why defining raw data contracts for sources is important in a data pipeline.

Think about what happens if bad data enters your system.

Describe what kind of rules might be included in a raw data contract.

Consider how you would check if data is 'correct' before using it.