Why sources define raw data contracts in dbt - Visual Breakdown

Concept Flow - Why sources define raw data contracts

Raw Data Source

↓

Define Raw Data Contract

↓

Set Expectations: Schema, Types, Quality

↓

Data Consumers Use Contract

↓

Detect Changes or Errors Early

↓

Maintain Data Reliability

This flow shows how defining raw data contracts sets clear rules for raw data, helping users trust and use data safely.

Execution Sample

dbt

sources:
  - name: raw_sales
    tables:
      - name: transactions
        freshness:
          warn_after:
            count: 24
            period: hour

This dbt source config defines a raw data contract for the 'transactions' table with freshness expectations.

Execution Table

Step	Action	Evaluation	Result
1	Define source 'raw_sales.transactions'	Set schema and freshness rules	Contract established for raw data
2	Data pipeline loads raw data	Check data against contract	Data matches schema and freshness
3	Data consumer queries source	Uses contract to trust data	Reliable data used in models
4	Raw data changes unexpectedly	Contract detects schema or freshness violation	Alert triggered for investigation
5	Fix data or update contract	Restore contract compliance	Data reliability maintained
6	End	All checks passed or issues resolved	Data pipeline stable

💡 Execution stops when data contract is either met or violations are detected and addressed

Variable Tracker

Variable	Start	After Step 2	After Step 4	Final
Data Contract Status	Not defined	Active and valid	Violation detected	Restored or updated

Key Moments - 2 Insights

Why do we define a raw data contract instead of just trusting the raw data?

Defining a contract sets clear expectations for schema and freshness, so any unexpected changes or errors are caught early, as shown in step 4 of the execution table.

What happens if the raw data changes but the contract is not updated?

The contract detects violations and triggers alerts (step 4), preventing unreliable data from being used downstream.

Visual Quiz - 3 Questions

Look at the execution table, what is the result after step 3?

A Data consumer uses unreliable data

B Contract is violated

C Data consumer uses reliable data

D Data pipeline stops

💡 Hint

Check the 'Result' column in row for step 3 in the execution table

At which step does the contract detect a violation?

A Step 4

B Step 2

C Step 3

D Step 5

💡 Hint

Look for 'Violation detected' in the 'Evaluation' column in the execution table

If the contract was never defined, what would be the status at 'After Step 2' in variable_tracker?

A Violation detected

B Not defined

C Active and valid

D Restored or updated

💡 Hint

Refer to the 'Data Contract Status' row in variable_tracker for the initial state

Concept Snapshot

Define raw data contracts in dbt sources to set clear rules on schema and freshness.
Contracts help detect data issues early.
They ensure data consumers trust raw data.
Violations trigger alerts for quick fixes.
Maintains overall data reliability.

Full Transcript

In dbt, defining raw data contracts means setting clear expectations for raw data sources, such as schema and freshness rules. This helps catch any unexpected changes or errors early. The flow starts with defining the contract, then loading data, checking it against the contract, and using it reliably downstream. If data changes unexpectedly, the contract detects violations and triggers alerts. Fixing data or updating the contract restores reliability. This process ensures data consumers can trust raw data and maintain stable pipelines.