Why documentation makes data discoverable in dbt - See It in Action

Why documentation makes data discoverable

📖 Scenario: You work in a team that builds data models using dbt. Your team wants to make it easy for everyone to find and understand the data models you create. Good documentation helps with this by explaining what each model does and how to use it.

🎯 Goal: Create a simple dbt model and add documentation to it. Then, configure dbt to generate documentation so that the data models become easy to discover and understand for your team.

📋 What You'll Learn

Create a dbt model file with a simple SQL query

Add a description to the model in the schema.yml file

Configure dbt to generate documentation

Run dbt commands to build and serve the documentation

💡 Why This Matters

🌍 Real World

Teams use dbt documentation to help everyone understand and find data models quickly, improving collaboration and reducing confusion.

💼 Career

Data analysts and engineers often need to document their work so others can trust and use data effectively. Knowing how to document in dbt is a valuable skill.

1

Create a simple dbt model

Create a dbt model file named my_first_model.sql inside the models folder. Write a simple SQL query that selects all columns from the raw_data.customers table.

dbt

-- Create the file models/my_first_model.sql
-- Write the SQL query below

-- Your code here

Hint

Use a simple SELECT * FROM raw_data.customers query in the model file.

2

Add documentation to the model

Create or update the schema.yml file in the models folder. Add a description for the model my_first_model under the models section. Use the description: 'This model selects all customer data from the raw source.'

dbt

select * from raw_data.customers

# Add a schema.yml file with model description
# Your code here

Hint

In schema.yml, add a models list with an entry for my_first_model and a description.

3

Configure dbt to generate documentation

In your dbt project, ensure you have a dbt_project.yml file. Add or confirm the docs section exists with generate: true to enable documentation generation.

dbt

select * from raw_data.customers

models:
  - name: my_first_model
    description: 'This model selects all customer data from the raw source.'

# Add or confirm docs section in dbt_project.yml
# Your code here

Hint

In dbt_project.yml, add a docs section with generate: true to enable documentation.

4

Build and serve the documentation

Run the dbt commands dbt docs generate and dbt docs serve in your terminal to build and view the documentation. This will make your model and its description discoverable in a web browser.

dbt

select * from raw_data.customers

models:
  - name: my_first_model
    description: 'This model selects all customer data from the raw source.'

# dbt_project.yml content
name: 'my_dbt_project'
version: '1.0'
config-version: 2

profile: 'default'

source-paths: ['models']

analysis-paths: ['analysis']

test-paths: ['tests']

data-paths: ['data']

macro-paths: ['macros']

docs:
  generate: true

# Run dbt docs generate and dbt docs serve commands
# Your code here

Hint

Run dbt docs generate to build docs and dbt docs serve to open them in your browser.

Practice

1. Why is documentation important in dbt projects for data discoverability?

easy

A. It speeds up the data processing time.

B. It explains data clearly so users can find and understand it easily.

C. It automatically fixes errors in data models.

D. It encrypts data for security.

Solution

1. Step 1: Understand the purpose of documentation in dbt

   Documentation provides clear explanations about data models and columns.

2. Step 2: Connect documentation to data discoverability

   Clear explanations help users find and understand data easily, improving discoverability.

3. Final Answer:

   It explains data clearly so users can find and understand it easily. -> Option B

4. Quick Check:

   Documentation improves discoverability [OK]

Hint: Documentation means clear explanations for easy data finding [OK]

Common Mistakes:

• Confusing documentation with data processing speed
• Thinking documentation fixes data errors automatically
• Assuming documentation encrypts data

2. Which of the following is the correct way to add a description to a dbt model in YAML?

easy

A. models:
- name: sales
description: 'Contains sales data by region'

B. models:
name: sales
description: 'Contains sales data by region'

C. model:
- name: sales
description: 'Contains sales data by region'

D. models:
- sales:
description: 'Contains sales data by region'

Solution

1. Step 1: Recall YAML structure for dbt model descriptions

   The correct syntax uses 'models:' followed by a list with '- name:' and 'description:' keys.

2. Step 2: Identify the option matching this structure

   models: - name: sales description: 'Contains sales data by region' correctly uses a list item with 'name' and 'description' under 'models'.

3. Final Answer:

   models:\n - name: sales\n description: 'Contains sales data by region' -> Option A

4. Quick Check:

   Correct YAML list syntax [OK]

Hint: YAML lists use dash and indentation for model descriptions [OK]

Common Mistakes:

• Missing dash for list items
• Using singular 'model' instead of 'models'
• Incorrect indentation breaking YAML format

3. Given this YAML snippet in a dbt model file:

models:
  - name: customers
    description: 'Customer details including name and email'
  - name: orders
    description: 'Order records with dates and amounts'

What will dbt documentation show for the 'orders' model?

medium

A. Error loading description

B. Customer details including name and email

C. Order records with dates and amounts

D. No description available

Solution

1. Step 1: Locate the 'orders' model in the YAML snippet

   The 'orders' model is listed with a description: 'Order records with dates and amounts'.

2. Step 2: Understand dbt documentation usage

   dbt uses the description text to show model info in docs.

3. Final Answer:

   Order records with dates and amounts -> Option C

4. Quick Check:

   Model description matches YAML text [OK]

Hint: Match model name to its description in YAML [OK]

Common Mistakes:

• Mixing descriptions between models
• Assuming missing description means error
• Confusing model names

4. You wrote this YAML for a dbt model description but the docs show no description:

models:
  name: products
  description: 'Product catalog details'

What is the likely error?

medium

A. Missing dash (-) before 'name' to define list item

B. Incorrect key 'description' instead of 'desc'

C. YAML does not support descriptions

D. Model name should be uppercase

Solution

1. Step 1: Check YAML list syntax for models

   dbt expects 'models:' followed by a list indicated by '-'. Missing dash means no list item.

2. Step 2: Identify the missing dash before 'name'

   Without '-', YAML treats 'name' as a key under 'models', not a list item, so description is ignored.

3. Final Answer:

   Missing dash (-) before 'name' to define list item -> Option A

4. Quick Check:

   Dash defines list items in YAML [OK]

Hint: Always use dash for list items in YAML [OK]

Common Mistakes:

• Using wrong key names
• Thinking YAML disallows descriptions
• Ignoring YAML indentation rules

5. You want to improve data discoverability by adding descriptions to columns in a dbt model. Which YAML snippet correctly documents the 'customer_id' column with a description?

hard

A. models:
- name: customers
columns:
- name: customer_id
desc: 'Unique ID for each customer'

B. models:
- name: customers
columns:
customer_id: 'Unique ID for each customer'

C. models:
- name: customers
columns:
- customer_id: 'Unique ID for each customer'

D. models:
- name: customers
columns:
- name: customer_id
description: 'Unique ID for each customer'

Solution

1. Step 1: Recall correct YAML structure for column documentation in dbt

   Columns are listed as items with '- name:' and 'description:' keys.

2. Step 2: Identify the option matching this structure

   models: - name: customers columns: - name: customer_id description: 'Unique ID for each customer' correctly uses '- name: customer_id' and 'description' key.

3. Final Answer:

   models:\n - name: customers\n columns:\n - name: customer_id\n description: 'Unique ID for each customer' -> Option D

4. Quick Check:

   Correct column description syntax [OK]

Hint: Use '- name:' and 'description:' for columns in YAML [OK]

Common Mistakes:

• Using key-value pairs without dash for columns
• Using 'desc' instead of 'description'
• Incorrect indentation breaking YAML