Organizing models in directories in dbt - Time & Space Complexity

Time Complexity: Organizing models in directories

O(n)

Understanding Time Complexity

When organizing dbt models in directories, it's important to understand how this affects the time it takes to run your project.

We want to know how the number of models and their folder structure impact execution time.

Scenario Under Consideration

Analyze the time complexity of this dbt project structure snippet.

models/
  sales/
    orders.sql
    customers.sql
  marketing/
    campaigns.sql
    leads.sql
  finance/
    revenue.sql
    expenses.sql

This structure organizes models into folders by domain, each containing multiple SQL files representing models.

Identify Repeating Operations

Identify the loops, recursion, array traversals that repeat.

• Primary operation: dbt runs each model file once during compilation and execution.
• How many times: Once per model file, so total runs equal the number of model files.

How Execution Grows With Input

As you add more model files in directories, the total execution time grows roughly in proportion to the number of models.

Input Size (n)	Approx. Operations
10 models	10 executions
100 models	100 executions
1000 models	1000 executions

Pattern observation: The time grows linearly as you add more models, regardless of how they are grouped in directories.

Final Time Complexity

Time Complexity: O(n)

This means the total time to run your dbt models grows directly with the number of models you have.

Common Mistake

[X] Wrong: "Organizing models into many folders will make dbt run faster because it processes folders separately."

[OK] Correct: dbt runs each model file individually regardless of folder structure, so folders do not reduce total execution time.

Interview Connect

Understanding how project structure affects execution helps you design scalable dbt projects and communicate clearly about performance.

Self-Check

What if we added model dependencies that require models to run in sequence? How would that affect the time complexity?

Practice

1. Why is it helpful to organize dbt models into directories?

easy

A. It keeps the project clean and easier to manage.

B. It makes dbt run faster.

C. It prevents errors in SQL syntax.

D. It automatically creates dashboards.

Solution

1. Step 1: Understand project organization benefits

   Organizing files into folders helps keep things tidy and easy to find.

2. Step 2: Relate to dbt model management

   In dbt, directories group models logically, making the project easier to manage.

3. Final Answer:

   It keeps the project clean and easier to manage. -> Option A

4. Quick Check:

   Organizing models = easier management [OK]

Hint: Folders group models logically for clarity [OK]

Common Mistakes:

• Thinking folders speed up dbt runs
• Believing folders fix SQL errors
• Assuming folders create dashboards automatically

2. Which of the following is the correct way to reference a model in a subdirectory in dbt SQL?

easy

A. SELECT * FROM subfolder-model_name

B. SELECT * FROM model_name.subfolder

C. SELECT * FROM subfolder.model_name

D. SELECT * FROM model_name

Solution

1. Step 1: Understand dbt model referencing

   dbt uses dot notation to reference models in subfolders: folder.model_name.

2. Step 2: Check each option

   Only SELECT * FROM subfolder.model_name uses correct dot notation with folder before model name.

3. Final Answer:

   SELECT * FROM subfolder.model_name -> Option C

4. Quick Check:

   Use folder.model_name to reference subfolder models [OK]

Hint: Use dot notation: folder.model_name [OK]

Common Mistakes:

• Reversing folder and model name order
• Using hyphens instead of dots
• Ignoring folder name when referencing

3. Given this dbt project structure:

models/
  sales/
    orders.sql
  marketing/
    campaigns.sql

What is the correct way to select all columns from the orders model inside the sales folder?

medium

A. SELECT * FROM orders

B. SELECT * FROM sales.orders

C. SELECT * FROM marketing.campaigns

D. SELECT * FROM models.sales.orders

Solution

1. Step 1: Identify model location

   The orders model is inside the sales folder under models.

2. Step 2: Use correct reference syntax

   dbt references models using folder.model_name, so sales.orders is correct.

3. Final Answer:

   SELECT * FROM sales.orders -> Option B

4. Quick Check:

   Reference subfolder models as folder.model_name [OK]

Hint: Use folder.model_name to select subfolder models [OK]

Common Mistakes:

• Omitting folder name when selecting
• Using full path with 'models.' prefix
• Selecting from wrong folder model

4. You have this dbt model file path: models/finance/revenue.sql. You wrote this SQL:

SELECT * FROM revenue

Why might this cause an error when running dbt?

medium

A. Because the model should be referenced as finance.revenue.

B. Because revenue.sql file is missing.

C. Because SELECT * FROM revenue is invalid SQL syntax.

D. Because dbt does not support subfolders.

Solution

1. Step 1: Understand model referencing in subfolders

   Models inside subfolders must be referenced with folder prefix, e.g., finance.revenue.

2. Step 2: Identify cause of error

   Using just revenue ignores the folder, causing dbt to not find the model.

3. Final Answer:

   Because the model should be referenced as finance.revenue. -> Option A

4. Quick Check:

   Reference subfolder models with folder prefix [OK]

Hint: Always prefix model with folder name in references [OK]

Common Mistakes:

• Assuming model name alone works in subfolders
• Thinking SQL syntax is wrong
• Believing dbt disallows subfolders

5. You want to organize your dbt models by year and quarter. Which directory structure best supports this while keeping references simple?

hard

A. models/year_2024_q1_sales.sql and reference as year_2024_q1_sales

B. models/q1/year_2024/sales.sql and reference as q1.year_2024.sales

C. models/sales/year_2024_q1.sql and reference as sales.year_2024_q1

D. models/year_2024/q1/sales.sql and reference as year_2024.q1.sales

Solution

1. Step 1: Understand multi-level directory organization

   Organizing by year then quarter creates clear hierarchy: year_2024/q1/.

2. Step 2: Check referencing simplicity

   dbt references nested folders with dot notation: year_2024.q1.sales is clear and logical.

3. Final Answer:

   models/year_2024/q1/sales.sql and reference as year_2024.q1.sales -> Option D

4. Quick Check:

   Use nested folders with dot notation references [OK]

Hint: Use nested folders matching dot notation references [OK]

Common Mistakes:

• Reversing folder order causing confusing references
• Using flat filenames losing hierarchy clarity
• Ignoring dot notation for nested folders