Documenting models in YAML in dbt - Time & Space Complexity

Time Complexity: Documenting models in YAML

O(n)

Understanding Time Complexity

We want to understand how the time it takes to document models in YAML grows as the number of models increases.

How does adding more models affect the work done by dbt when reading documentation?

Scenario Under Consideration

Analyze the time complexity of the following YAML documentation snippet for dbt models.

version: 2
models:
  - name: customers
    description: "Contains customer details"
    columns:
      - name: id
        description: "Unique customer ID"
      - name: name
        description: "Customer full name"
  - name: orders
    description: "Contains order records"
    columns:
      - name: order_id
        description: "Unique order ID"
      - name: customer_id
        description: "ID of the customer who placed the order"

This YAML documents two models, each with columns and descriptions.

Identify Repeating Operations

Look at what repeats when dbt processes this YAML documentation.

• Primary operation: Reading each model and its columns to build documentation.
• How many times: Once per model, and once per column inside each model.

How Execution Grows With Input

As the number of models and columns grows, dbt reads more entries.

Input Size (models)	Approx. Operations
10	Reads about 10 models and their columns
100	Reads about 100 models and their columns
1000	Reads about 1000 models and their columns

Pattern observation: The work grows roughly in direct proportion to the number of models and columns.

Final Time Complexity

Time Complexity: O(n)

This means the time to process documentation grows linearly with the number of models and columns.

Common Mistake

[X] Wrong: "Adding more models won't affect processing time much because YAML is just text."

[OK] Correct: Even though YAML is text, dbt must read and parse each model and column, so more models mean more work.

Interview Connect

Understanding how processing time grows with input size helps you explain efficiency in real projects, showing you can think about scaling and performance.

Self-Check

"What if we added nested descriptions or tests inside each model? How would the time complexity change?"

Practice

1. What is the main purpose of documenting models in YAML in a dbt project?

easy

A. To write SQL queries inside YAML files

B. To execute dbt models automatically

C. To add clear descriptions for models and columns to improve understanding

D. To store raw data files

Solution

1. Step 1: Understand the role of YAML documentation

   YAML files in dbt are used to add metadata like descriptions, not to run code or store data.

2. Step 2: Identify the benefit of documentation

   Adding descriptions for models and columns helps team members understand the data and maintain the project easily.

3. Final Answer:

   To add clear descriptions for models and columns to improve understanding -> Option C

4. Quick Check:

   Documentation purpose = Add descriptions [OK]

Hint: Documentation in YAML means adding descriptions, not code [OK]

Common Mistakes:

• Thinking YAML runs SQL code
• Confusing YAML with data storage
• Ignoring the importance of descriptions

2. Which of the following is the correct way to start documenting a model named orders in a YAML file?

easy

A. models:
orders
description: 'Contains order details'

B. model:
name: orders
description: 'Contains order details'

C. models:
- orders:
description: 'Contains order details'

D. models:
- name: orders
description: 'Contains order details'

Solution

1. Step 1: Recall YAML syntax for dbt model documentation

   dbt expects a list under models: with each model as a dictionary containing name and description.

2. Step 2: Match the correct structure

   models: - name: orders description: 'Contains order details' correctly uses a list with a dictionary having name and description. Other options misuse keys or structure.

3. Final Answer:

   models: - name: orders description: 'Contains order details' -> Option D

4. Quick Check:

   Model list with name and description = models: - name: orders description: 'Contains order details' [OK]

Hint: Use dash (-) for list items under models in YAML [OK]

Common Mistakes:

• Using singular 'model' instead of 'models'
• Not using dash for list items
• Incorrect indentation or key names

3. Given this YAML snippet documenting a model and its columns:

models:
  - name: customers
    description: 'Customer information'
    columns:
      - name: id
        description: 'Unique customer ID'
      - name: email
        description: 'Customer email address'

What will dbt show as the description for the email column?

medium

A. Unique customer ID

B. Customer email address

C. Customer information

D. No description

Solution

1. Step 1: Locate the column description in YAML

   The email column is listed under columns with its own description key.

2. Step 2: Identify the description text for the email column

   The description for email is 'Customer email address', which dbt will display for that column.

3. Final Answer:

   Customer email address -> Option B

4. Quick Check:

   Column description matches YAML text [OK]

Hint: Column descriptions are under columns > name in YAML [OK]

Common Mistakes:

• Confusing model description with column description
• Missing indentation causing YAML parsing errors
• Assuming no description if not repeated

4. You wrote this YAML to document a model but dbt throws an error:

models:
  - name: sales
    description: 'Sales data'
    columns:
      name: amount
      description: 'Sale amount'

What is the error in this YAML?

medium

A. Missing dash (-) before column name and description

B. Incorrect model name key

C. Description should be under models, not columns

D. YAML does not support nested lists

Solution

1. Step 1: Check YAML list syntax for columns

   Each column should be a list item with a dash (-) before its dictionary of keys.

2. Step 2: Identify missing dash in columns

   The name and description keys under columns lack the dash, so YAML treats them as keys of columns instead of list items.

3. Final Answer:

   Missing dash (-) before column name and description -> Option A

4. Quick Check:

   List items need dash (-) in YAML [OK]

Hint: Use dash (-) before each column in columns list [OK]

Common Mistakes:

• Forgetting dash for list items
• Misplacing description keys
• Confusing YAML lists and dictionaries

5. You want to document two models, users and transactions, each with columns and descriptions. Which YAML structure correctly documents both models with their columns?

hard

A. models:
- name: users
description: 'User data'
columns:
- name: user_id
description: 'User identifier'
- name: transactions
description: 'Transaction data'
columns:
- name: transaction_id
description: 'Transaction identifier'

B. models:
users:
description: 'User data'
columns:
user_id: 'User identifier'
transactions:
description: 'Transaction data'
columns:
transaction_id: 'Transaction identifier'

C. models:
- users:
description: 'User data'
columns:
- user_id: 'User identifier'
- transactions:
description: 'Transaction data'
columns:
- transaction_id: 'Transaction identifier'

D. models:
name: users
description: 'User data'
columns:
- name: user_id
description: 'User identifier'
name: transactions
description: 'Transaction data'
columns:
- name: transaction_id
description: 'Transaction identifier'

Solution

1. Step 1: Understand YAML list structure for multiple models

   dbt expects models as a list of dictionaries, each with name, description, and columns as a list.

2. Step 2: Evaluate each option's structure

   models: - name: users description: 'User data' columns: - name: user_id description: 'User identifier' - name: transactions description: 'Transaction data' columns: - name: transaction_id description: 'Transaction identifier' correctly uses a list with two model dictionaries, each with proper keys and column lists. The other options misuse keys or structure. models: name: users description: 'User data' columns: - name: user_id description: 'User identifier' name: transactions description: 'Transaction data' columns: - name: transaction_id description: 'Transaction identifier' repeats keys incorrectly.

3. Final Answer:

   models: - name: users description: 'User data' columns: - name: user_id description: 'User identifier' - name: transactions description: 'Transaction data' columns: - name: transaction_id description: 'Transaction identifier' -> Option A

4. Quick Check:

   Multiple models as list items with name and columns = models: - name: users description: 'User data' columns: - name: user_id description: 'User identifier' - name: transactions description: 'Transaction data' columns: - name: transaction_id description: 'Transaction identifier' [OK]

Hint: List each model with dash (-) and include columns as lists [OK]

Common Mistakes:

• Using model names as keys instead of list items
• Repeating keys at same level
• Not using dash for multiple models