Why governance ensures data trust in dbt - Visual Breakdown

Concept Flow - Why governance ensures data trust

Define Data Governance Rules

↓

Apply Rules to Data Pipelines

↓

Monitor Data Quality & Compliance

↓

Identify & Fix Issues

↓

Ensure Data is Reliable & Trusted

Data governance sets rules, applies them to data, monitors quality, fixes issues, and ensures data trust.

Execution Sample

dbt

version: 2
models:
  - name: sales
    columns:
      - name: id
        tests:
          - unique
          - not_null
      - name: status
        tests:
          - accepted_values:
              values: ["completed", "pending", "cancelled"]

This dbt model config applies tests to ensure data quality and trust.

Execution Table

Step	Action	Rule Applied	Result	Data Trust Impact
1	Define tests in schema.yml	unique, not_null, accepted_values	Rules ready to apply	Foundation for trust
2	Run dbt tests on sales model	unique on id	Pass: all ids unique	Ensures no duplicates
3	Run dbt tests on sales model	not_null on id	Pass: no null ids	Ensures completeness
4	Run dbt tests on sales model	accepted_values on status	Fail: found 'in_progress'	Detects invalid data
5	Fix data issue: update 'in_progress' to 'pending'	Data correction	Pass on re-test	Restores trust
6	Monitor ongoing tests	Continuous checks	All tests passing	Maintains trust over time

💡 All tests pass after fixing invalid data, ensuring trusted data

Variable Tracker

Variable	Start	After Step 2	After Step 4	After Step 5	Final
Data Quality Status	Undefined	Unique test passed	Accepted values test failed	Data corrected	All tests passed

Key Moments - 3 Insights

Why is it important to define tests before running them?

Defining tests sets clear rules to check data quality, as shown in Step 1 of the execution_table.

What happens when a test fails during data validation?

A failure, like in Step 4, signals bad data that must be fixed to restore trust.

How does fixing data issues affect data trust?

Fixing issues (Step 5) makes tests pass again, which rebuilds and maintains trust.

Visual Quiz - 3 Questions

Look at the execution_table, what test failed at Step 4?

A unique on id

B not_null on id

C accepted_values on status

D no test failed

💡 Hint

Check the 'Rule Applied' and 'Result' columns at Step 4 in execution_table

According to variable_tracker, what is the data quality status after Step 5?

A Unique test passed

B Data corrected

C Accepted values test failed

D All tests passed

💡 Hint

Look at the 'After Step 5' column for 'Data Quality Status' in variable_tracker

If the data issue was not fixed at Step 5, what would happen to data trust?

A Data trust would be lost due to failing tests

B Data trust would remain high

C Data trust would improve automatically

D Data trust is unrelated to tests

💡 Hint

Refer to Step 4 failure and Step 5 fix impact in execution_table

Concept Snapshot

Data governance means setting rules to check data quality.
Use tests like unique, not_null, and accepted_values.
Run tests regularly to find bad data.
Fix issues quickly to keep data trusted.
Trusted data helps teams make good decisions.

Full Transcript

Data governance ensures data trust by defining clear rules to check data quality. In dbt, these rules are tests like unique, not_null, and accepted_values. The process starts by defining tests, then running them on data models. If tests pass, data is trusted. If tests fail, it shows data problems that must be fixed. Fixing issues makes tests pass again, restoring trust. Continuous monitoring keeps data reliable over time. This step-by-step approach builds confidence that data is accurate and ready for use.

Practice

1. Why is data governance important for building trust in data within a team?

easy

A. It removes the need for data documentation.

B. It makes data processing slower and more complex.

C. It hides data from most team members to avoid confusion.

D. It sets clear rules to keep data accurate and consistent.

Solution

1. Step 1: Understand the role of governance

   Governance creates rules and standards to ensure data is accurate and consistent.

2. Step 2: Connect governance to trust

   When data is accurate and consistent, teams can trust and confidently use it.

3. Final Answer:

   It sets clear rules to keep data accurate and consistent. -> Option D

4. Quick Check:

   Governance = Clear rules = Trust [OK]

Hint: Governance means clear rules for accurate data [OK]

Common Mistakes:

• Thinking governance slows down data use
• Believing governance hides data
• Assuming governance removes documentation

2. Which dbt feature directly supports data governance by checking data quality?

easy

A. dbt models

B. dbt tests

C. dbt snapshots

D. dbt seeds

Solution

1. Step 1: Identify dbt features

   dbt models build data tables, seeds load static data, snapshots track changes, tests check data quality.

2. Step 2: Match feature to governance

   Data governance needs quality checks, which dbt tests provide by validating data correctness.

3. Final Answer:

   dbt tests -> Option B

4. Quick Check:

   Tests = Quality checks = Governance support [OK]

Hint: Tests check data quality to support governance [OK]

Common Mistakes:

• Confusing models with tests
• Thinking snapshots check quality
• Assuming seeds validate data

3. Given this dbt test code snippet, what will be the result if the column user_id contains NULL values?

version: 2
models:
  - name: users
    columns:
      - name: user_id
        tests:
          - not_null

medium

A. The test will fail because NULL values exist in user_id.

B. The test will pass because NULLs are allowed.

C. The test will be ignored by dbt.

D. The test will cause a syntax error.

Solution

1. Step 1: Understand the not_null test

   This test checks that the specified column has no NULL values.

2. Step 2: Analyze the presence of NULLs

   If user_id contains NULLs, the test will detect them and fail.

3. Final Answer:

   The test will fail because NULL values exist in user_id. -> Option A

4. Quick Check:

   not_null test + NULL values = Fail [OK]

Hint: not_null test fails if any NULL found [OK]

Common Mistakes:

• Assuming NULLs pass not_null test
• Thinking test is ignored automatically
• Believing syntax error occurs

4. You wrote this dbt test to ensure no duplicate emails:

tests:
  - unique:
      column_name: email

But it does not run. What is the error?

medium

A. The test name should be unique without parameters.

B. The test should be written as - unique without column_name.

C. The test should be under the columns section with just - unique.

D. The test requires a SQL file, so this YAML is invalid.

Solution

1. Step 1: Check correct test syntax in dbt schema.yml

   Tests on columns go under the columns section with just - unique listed.

2. Step 2: Identify the mistake

   The test is incorrectly placed and uses column_name, which is not valid here.

3. Final Answer:

   The test should be under the columns section with just - unique. -> Option C

4. Quick Check:

   Unique test syntax = under columns with - unique [OK]

Hint: Place tests under columns with simple names [OK]

Common Mistakes:

• Using parameters like column_name in schema tests
• Placing tests outside columns section
• Thinking SQL files are needed for built-in tests

5. Your team wants to ensure data trust by documenting all models and adding tests in dbt. Which combination best supports governance?

hard

A. Write detailed model descriptions and add tests like not_null and unique on key columns.

B. Only add tests without documentation to save time.

C. Document models but skip tests since they slow down builds.

D. Use snapshots only to track data changes without tests or docs.

Solution

1. Step 1: Understand governance needs

   Governance requires both clear documentation and data quality checks to build trust.

2. Step 2: Evaluate options

   Write detailed model descriptions and add tests like not_null and unique on key columns. combines documentation and tests, supporting governance best. Others skip key parts.

3. Final Answer:

   Write detailed model descriptions and add tests like not_null and unique on key columns. -> Option A

4. Quick Check:

   Docs + tests = Strong governance [OK]

Hint: Combine docs and tests for best governance [OK]

Common Mistakes:

• Skipping docs or tests thinking one is enough
• Avoiding tests to speed builds
• Relying only on snapshots for trust