Why dbt project structure? - Purpose & Use Cases

The Big Idea

What if your messy SQL files could magically organize themselves and run perfectly every time?

The Scenario

Imagine you have many SQL files scattered in different folders without any clear order. You want to build a data model, but you keep losing track of which file does what and how they connect.

The Problem

Manually managing SQL files and dependencies is slow and confusing. You might run the wrong queries, overwrite files, or miss important steps. It's easy to make mistakes and hard to fix them.

The Solution

The dbt project structure organizes your files clearly: models, tests, macros, and data sources each have their place. This structure helps dbt understand how everything fits together and runs your data transformations smoothly.

Before vs After

✗ Before

run each SQL file separately without order
-- no clear folder structure

✓ After

dbt run
-- uses project folders and config to run models in order

What It Enables

With a clear dbt project structure, you can build reliable, maintainable data pipelines that run automatically and scale easily.

Real Life Example

A data analyst working on sales data can organize raw tables, cleaned models, and reports in separate folders. dbt runs them in the right order, so the analyst gets fresh, accurate dashboards every day without manual work.

Key Takeaways

Manual SQL file management is confusing and error-prone.

dbt project structure organizes files for clarity and automation.

This structure enables smooth, reliable data transformations.

Practice

1. Which folder in a dbt project typically contains SQL files that define your data transformations?

easy

A. snapshots/

B. models/

C. macros/

D. tests/

Solution

1. Step 1: Understand folder purposes

   The models/ folder is where SQL files for data transformations live.

2. Step 2: Identify correct folder for SQL models

   Other folders like macros/ hold reusable code, snapshots/ hold snapshot definitions, and tests/ hold test files.

3. Final Answer:

   models/ -> Option B

4. Quick Check:

   Data transformations = models/ folder [OK]

Hint: Models folder holds SQL transformations [OK]

Common Mistakes:

• Confusing macros/ with models/
• Thinking snapshots/ holds models
• Assuming tests/ contains SQL models

2. Which of the following is the correct name for the main configuration file in a dbt project?

easy

A. dbt_project.yml

B. dbt_config.yml

C. project.yaml

D. dbt_settings.yml

Solution

1. Step 1: Recall main config file name

   The main configuration file for dbt projects is named dbt_project.yml.

2. Step 2: Verify other options

   Other options like dbt_config.yml or dbt_settings.yml are incorrect names.

3. Final Answer:

   dbt_project.yml -> Option A

4. Quick Check:

   Main config file = dbt_project.yml [OK]

Hint: Main config file always named dbt_project.yml [OK]

Common Mistakes:

• Using dbt_config.yml instead
• Confusing with generic project.yaml
• Assuming settings file controls project

3. Given this dbt project structure snippet:

my_dbt_project/
├── models/
│   ├── customers.sql
│   └── orders.sql
├── macros/
│   └── date_utils.sql
└── dbt_project.yml

Which file would you edit to add a reusable SQL function?

medium

A. models/customers.sql

B. dbt_project.yml

C. macros/date_utils.sql

D. models/orders.sql

Solution

1. Step 1: Identify purpose of macros/ folder

   The macros/ folder holds reusable SQL functions and macros.

2. Step 2: Locate reusable function file

   The file macros/date_utils.sql is the right place to add reusable SQL functions.

3. Final Answer:

   macros/date_utils.sql -> Option C

4. Quick Check:

   Reusable SQL functions = macros/ folder [OK]

Hint: Reusable SQL code goes in macros/ folder [OK]

Common Mistakes:

• Adding functions inside models/ files
• Editing dbt_project.yml for SQL code
• Confusing macros/ with models/

4. You see this error when running dbt: Compilation Error: Could not find model 'sales_summary'. Which is the most likely cause related to project structure?

medium

A. The 'sales_summary.sql' file is missing from the models/ folder.

B. The 'sales_summary.sql' file is inside the macros/ folder.

C. The dbt_project.yml file is missing.

D. The snapshots/ folder contains 'sales_summary.sql'.

Solution

1. Step 1: Understand error meaning

   The error means dbt cannot find the model named 'sales_summary'.

2. Step 2: Check model file location

   Models must be in the models/ folder. If the file is missing there, dbt can't compile it.

3. Final Answer:

   The 'sales_summary.sql' file is missing from the models/ folder. -> Option A

4. Quick Check:

   Missing model file in models/ causes compilation error [OK]

Hint: Models must be in models/ folder to compile [OK]

Common Mistakes:

• Placing model files in macros/ folder
• Assuming missing dbt_project.yml causes this error
• Confusing snapshots/ with models/

5. You want to organize your dbt project so that models related to customers and orders are in separate folders inside models/. How should you update your dbt_project.yml to reflect this structure?

hard

A. Add models:
my_dbt_project:
+materialized: view only

B. No changes needed; dbt auto-detects subfolders without config

C. Add macros:
my_dbt_project:
customers:
+materialized: table

D. Add models:
my_dbt_project:
customers:
+materialized: table
orders:
+materialized: table

Solution

1. Step 1: Understand folder-specific config in dbt_project.yml

   You can specify configs per subfolder inside models/ by nesting them under your project name.

2. Step 2: Define separate configs for customers and orders folders

   Adding customers: and orders: keys with materialization settings applies configs to those subfolders.

3. Final Answer:

   Add models: my_dbt_project: customers: +materialized: table orders: +materialized: table -> Option D

4. Quick Check:

   Use nested keys in dbt_project.yml for subfolder configs [OK]

Hint: Use nested keys in dbt_project.yml for subfolder configs [OK]

Common Mistakes:

• Not nesting configs under project name
• Configuring macros instead of models
• Assuming no config needed for subfolders