Doc blocks for reusable descriptions in dbt - Deep Dive

Overview - Doc blocks for reusable descriptions

What is it?

Doc blocks in dbt are reusable pieces of text that describe models, columns, or other parts of your data project. They let you write a description once and use it in many places, keeping your documentation consistent and easy to update. Instead of repeating the same explanation multiple times, you reference the doc block wherever needed. This helps teams understand data better and saves time.

Why it matters

Without doc blocks, you would have to write the same descriptions over and over, which can lead to mistakes and outdated info. This makes it hard for people to trust or understand the data. Doc blocks solve this by centralizing descriptions, so when something changes, you update it once and all references stay accurate. This improves data quality and team communication.

Where it fits

Before learning doc blocks, you should know basic dbt concepts like models, columns, and how to write documentation in dbt. After mastering doc blocks, you can explore advanced documentation features like macros and automated docs generation. Doc blocks fit into the documentation and maintainability part of the dbt learning path.

Mental Model

Core Idea

Doc blocks are like reusable labels or notes you write once and attach everywhere to keep your data descriptions consistent and easy to maintain.

Think of it like...

Imagine you have a sticky note with a recipe tip that you want to share on many recipe cards. Instead of writing the tip on each card, you write it once on a sticky note and stick it on every card. If the tip changes, you update the sticky note, and all cards show the new tip instantly.

┌───────────────┐
│ Doc Block     │
│ (Reusable     │
│ Description)  │
└──────┬────────┘
       │
       ▼
┌───────────────┐   ┌───────────────┐   ┌───────────────┐
│ Model A       │   │ Model B       │   │ Model C       │
│ Uses Doc Block│   │ Uses Doc Block│   │ Uses Doc Block│
└───────────────┘   └───────────────┘   └───────────────┘

Build-Up - 6 Steps

1

Foundation

What are doc blocks in dbt

Concept: Doc blocks are named text blocks that store descriptions for reuse.

In dbt, you can define a doc block in your project under the 'docs' directory or inside your schema.yml files. A doc block looks like this: docs: my_description: description: "This is a reusable description for a column or model." You can then reference this doc block elsewhere using {{ doc('my_description') }}.

Result

You create a single source of truth for descriptions that can be reused.

Understanding that doc blocks separate content from usage helps keep documentation clean and consistent.

2

Foundation

Referencing doc blocks in schema files

Concept: You can insert doc blocks into model or column descriptions by referencing them.

In your schema.yml file, instead of writing a description directly, you use the doc function: models: - name: my_model description: "{{ doc('my_description') }}" columns: - name: my_column description: "{{ doc('my_description') }}" This pulls the text from the doc block into the documentation.

Result

Descriptions appear in the generated docs exactly as defined in the doc block.

Knowing how to reference doc blocks lets you avoid duplication and errors in descriptions.

3

Intermediate

Organizing doc blocks for large projects

Concept: Doc blocks can be organized in separate files for better management.

For bigger projects, you can create multiple doc files inside the 'docs' folder, each containing related descriptions: # docs/customer_docs.yml customer_name: description: "The full name of the customer." customer_id: description: "Unique identifier for each customer." Then reference them in schema.yml as usual with {{ doc('customer_name') }}.

Result

Documentation stays organized and easy to navigate as the project grows.

Structuring doc blocks by topic or domain helps teams find and update descriptions faster.

4

Intermediate

Combining doc blocks with Jinja templating

🤔Before reading on: do you think you can add variables or logic inside doc blocks? Commit to your answer.

Concept: Doc blocks support Jinja templating, allowing dynamic content inside descriptions.

You can use Jinja code inside doc blocks to insert variables or conditional text: # docs/status_docs.yml status_description: description: "The status can be {{ 'active' if is_active else 'inactive' }} depending on the flag." When rendering docs, dbt evaluates this code, making descriptions dynamic based on context.

Result

Descriptions can adapt automatically, reducing manual updates.

Understanding that doc blocks are Jinja templates unlocks powerful dynamic documentation capabilities.

5

Advanced

Using doc blocks for column-level consistency

🤔Before reading on: do you think doc blocks can help enforce consistent column descriptions across multiple models? Commit to yes or no.

Concept: Doc blocks enable consistent descriptions for columns used in many models or tables.

If you have a column like 'created_at' used in many models, define a doc block: created_at: description: "Timestamp when the record was created." Then in each model's schema.yml, reference it: columns: - name: created_at description: "{{ doc('created_at') }}" This ensures all 'created_at' columns share the exact same description.

Result

Documentation is uniform, reducing confusion and errors.

Knowing doc blocks enforce consistency prevents drift in documentation quality across large projects.

6

Expert

Advanced doc blocks with macros and variables

🤔Before reading on: can doc blocks call macros or accept parameters? Commit to your answer.

Concept: Doc blocks can include macros and accept parameters for highly flexible descriptions.

You can create macros that generate descriptions dynamically and call them inside doc blocks: # macros/description_macros.sql {% macro dynamic_description(column_name) %} Description for {{ column_name }} column. {% endmacro %} # docs/dynamic_docs.yml custom_desc: description: "{{ dynamic_description('order_id') }}" This lets you build descriptions that change based on input, making documentation smarter and more maintainable.

Result

Documentation adapts to complex needs without manual rewriting.

Understanding this unlocks the full power of dbt's templating system for documentation.

Under the Hood

Doc blocks are stored as YAML entries in dbt project files. When dbt builds documentation, it processes these YAML files and replaces references like {{ doc('name') }} with the stored text. This happens through Jinja templating, which evaluates the doc function and inserts the corresponding description. The system caches these blocks for efficiency and supports nested templating and macros.

Why designed this way?

dbt was designed to keep code and documentation DRY (Don't Repeat Yourself). Doc blocks centralize descriptions to avoid duplication and errors. Using Jinja templating allows flexibility and dynamic content. This design balances simplicity for beginners with power for advanced users, unlike static documentation that quickly becomes outdated.

┌───────────────┐
│ YAML Doc Block│
│ (docs/*.yml)  │
└──────┬────────┘
       │
       ▼
┌───────────────┐       ┌───────────────┐
│ Jinja Engine  │──────▶│ Rendered Text │
│ (doc('name')) │       │ Inserted in   │
└──────┬────────┘       │ schema.yml or │
       │                │ model docs     │
       ▼                └───────────────┘
┌───────────────┐
│ dbt Docs Site │
│ (HTML Output) │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think doc blocks automatically update descriptions in all models without referencing them? Commit to yes or no.

Common Belief: Doc blocks automatically update all descriptions everywhere without needing explicit references.

Reality: Doc blocks only update descriptions where you explicitly use the {{ doc('name') }} reference. If a model or column has a hardcoded description, it won't change automatically.

Why it matters: Assuming automatic updates can cause outdated or inconsistent documentation if references are missing.

Quick: Can doc blocks contain executable SQL code? Commit to yes or no.

Common Belief: Doc blocks can run SQL queries or code inside descriptions to fetch live data.

Reality: Doc blocks only support Jinja templating, not SQL execution. They cannot query databases or run code that returns data dynamically at runtime.

Why it matters: Expecting live data in docs can lead to confusion and misuse of doc blocks.

Quick: Do you think doc blocks can be nested inside each other? Commit to yes or no.

Common Belief: You can nest doc blocks by referencing one doc block inside another.

Reality: dbt does not support nesting doc blocks directly. Each doc block is a standalone text block referenced individually.

Why it matters: Trying to nest doc blocks can cause errors or unexpected documentation output.

Quick: Are doc blocks only useful for model descriptions? Commit to yes or no.

Common Belief: Doc blocks are only for describing entire models, not columns or other elements.

Reality: Doc blocks can be used for any documentation text, including columns, tests, or even macros.

Why it matters: Limiting doc blocks to models reduces their usefulness and misses opportunities for consistent documentation.

Expert Zone

1

Doc blocks can leverage Jinja macros to create parameterized, reusable descriptions that adapt to different contexts.

2

Using doc blocks reduces merge conflicts in documentation because descriptions are centralized rather than duplicated across many files.

3

Doc blocks support markdown formatting, allowing rich text, links, and lists inside descriptions for better readability.

When NOT to use

Doc blocks are not suitable when descriptions need to be unique per model or column instance. In such cases, write direct descriptions. Also, avoid doc blocks if you require live data or dynamic content that depends on database queries; use external documentation tools instead.

Production Patterns

In production dbt projects, teams create a 'docs' folder with categorized doc block files for domains like customers, orders, or finance. They enforce referencing doc blocks in all schema files via code reviews to maintain consistency. Advanced teams combine doc blocks with macros to generate descriptions that include environment-specific details or data freshness notes.

Connections

Software Documentation Templates

Doc blocks are similar to reusable templates in software docs that keep descriptions consistent across functions or modules.

Understanding doc blocks as templates helps appreciate their role in reducing duplication and errors in any documentation system.

Content Management Systems (CMS)

Doc blocks function like CMS content snippets that can be inserted in multiple pages and updated centrally.

Recognizing this connection shows how dbt documentation borrows ideas from web content management for maintainability.

Legal Contract Clauses

Doc blocks resemble standard contract clauses reused across many agreements to ensure consistency and reduce drafting errors.

Seeing doc blocks as legal clauses highlights the importance of reusability and accuracy in critical documents beyond tech.

Common Pitfalls

#1 Forgetting to reference doc blocks and writing descriptions directly.

Wrong approach:

columns:
- name: user_id
description: "Unique user identifier."

models:
- name: users
description: "Table of users."

Correct approach:

columns:
- name: user_id
description: "{{ doc('user_id_description') }}"

models:
- name: users
description: "{{ doc('users_table_description') }}"

Root cause:

Not understanding that doc blocks must be explicitly referenced to be effective.

#2 Trying to nest doc blocks inside each other.

Wrong approach:

docs:
parent_desc:
description: "See {{ doc('child_desc') }} for details."

child_desc:
description: "Child description text."

Correct approach:

docs:
parent_desc:
description: "Parent description text."

child_desc:
description: "Child description text."

Use separate references where needed instead of nesting.

Root cause:

Misunderstanding that doc blocks are standalone and cannot call other doc blocks.

#3 Including SQL code inside doc block descriptions expecting it to run.

Wrong approach:

docs:
sql_desc:
description: "The count is {{ run_query('SELECT COUNT(*) FROM table') }}."

Correct approach:

docs:
sql_desc:
description: "This column counts the number of records."

Root cause:

Confusing Jinja templating with SQL execution capabilities inside doc blocks.

Key Takeaways

Doc blocks in dbt let you write descriptions once and reuse them everywhere, keeping documentation consistent and easy to update.

You must explicitly reference doc blocks using {{ doc('name') }} to insert their content into models or columns.

Doc blocks support Jinja templating, enabling dynamic and flexible documentation content.

Organizing doc blocks in separate files helps manage large projects and improves team collaboration.

Advanced use of doc blocks with macros unlocks powerful, parameterized descriptions that adapt to complex needs.