Why project structure scales with team size in dbt - See It in Action

Why project structure scales with team size

📖 Scenario: You are part of a growing data team using dbt to manage data transformations. As more team members join, organizing your dbt project well becomes important to keep work clear and efficient.

🎯 Goal: Build a simple dbt project structure that shows how to organize models by team responsibility to help scale work as the team grows.

📋 What You'll Learn

Create a dictionary called models with keys as team names and values as lists of model names

Create a variable called team_to_focus and set it to the team name to analyze

Use a for loop with variables model to get all models for the selected team from models

Print the list of models for the selected team

💡 Why This Matters

🌍 Real World

In real data teams, organizing dbt models by team or function helps avoid confusion and merge conflicts as more people work on the project.

💼 Career

Data engineers and analysts often collaborate on dbt projects. Knowing how to structure projects for team scaling is a valuable skill.

1

Create the dbt project model structure

Create a dictionary called models with these exact entries: 'analytics': ['user_activity', 'sales_summary'], 'engineering': ['raw_events', 'staging_users'], 'marketing': ['campaign_performance', 'lead_scoring']

dbt

# Create the models dictionary with team keys and model lists
# Your code here

Hint

Use curly braces to create a dictionary. Keys are team names as strings, values are lists of model names.

2

Select the team to focus on

Create a variable called team_to_focus and set it to the string 'analytics'

dbt

models = {
    'analytics': ['user_activity', 'sales_summary'],
    'engineering': ['raw_events', 'staging_users'],
    'marketing': ['campaign_performance', 'lead_scoring']
}
# Set the team_to_focus variable to 'analytics'
# Your code here

Hint

Assign the string 'analytics' to the variable team_to_focus.

3

Get models for the selected team

Create a list called selected_models by using a for loop with variable model to iterate over models[team_to_focus] and collect each model

dbt

models = {
    'analytics': ['user_activity', 'sales_summary'],
    'engineering': ['raw_events', 'staging_users'],
    'marketing': ['campaign_performance', 'lead_scoring']
}

team_to_focus = 'analytics'

# Use a for loop to create selected_models list from models[team_to_focus]
# Your code here

Hint

Use a list comprehension with model iterating over models[team_to_focus].

4

Print the selected team's models

Write a print statement to display the selected_models list

dbt

models = {
    'analytics': ['user_activity', 'sales_summary'],
    'engineering': ['raw_events', 'staging_users'],
    'marketing': ['campaign_performance', 'lead_scoring']
}

team_to_focus = 'analytics'

selected_models = [model for model in models[team_to_focus]]

# Print the selected_models list
# Your code here

Hint

Use print(selected_models) to show the list of models.

Practice

1. Why is having a clear project structure important when the dbt team grows larger?

easy

A. It helps keep work organized and reduces confusion among team members.

B. It makes the project run faster on the computer.

C. It allows the team to use more complex SQL queries.

D. It automatically fixes errors in the data models.

Solution

1. Step 1: Understand team collaboration needs

   As team size increases, many people work on the same project, so organization is key to avoid confusion.

2. Step 2: Recognize the role of project structure

   A clear structure helps separate work, making it easier to manage and understand for everyone.

3. Final Answer:

   It helps keep work organized and reduces confusion among team members. -> Option A

4. Quick Check:

   Clear structure = organized teamwork [OK]

Hint: Think about teamwork and avoiding confusion [OK]

Common Mistakes:

• Confusing project speed with team organization
• Assuming structure fixes errors automatically
• Believing complexity equals better structure

2. Which of the following is the correct way to organize models in a dbt project for a growing team?

easy

A. Separate models into folders by their purpose or domain.

B. Use random folder names to avoid conflicts.

C. Write all models in a single SQL file.

D. Put all models in one folder to keep things simple.

Solution

1. Step 1: Review folder organization best practices

   Separating models by purpose or domain helps teams work independently without conflicts.

2. Step 2: Evaluate options

   Keeping all models in one place or random names causes confusion and conflicts.

3. Final Answer:

   Separate models into folders by their purpose or domain. -> Option A

4. Quick Check:

   Folder by purpose = less conflict [OK]

Hint: Group models by what they do, not all together [OK]

Common Mistakes:

• Thinking one folder is simpler for many people
• Ignoring the importance of clear naming
• Believing random names help avoid conflicts

3. Given this dbt project structure:

models/
  staging/
    customers.sql
    orders.sql
  marts/
    sales.sql
    finance.sql

What is the main benefit of this structure for a team?

medium

A. It forces everyone to edit the same files to avoid duplication.

B. It hides the staging models from the team.

C. It makes the project run faster by splitting files.

D. It allows team members to work on staging and marts independently.

Solution

1. Step 1: Analyze folder roles

   Staging folder holds raw data models; marts folder holds business logic models.

2. Step 2: Understand team workflow

   Separating these allows different team members to work on different layers without interfering.

3. Final Answer:

   It allows team members to work on staging and marts independently. -> Option D

4. Quick Check:

   Separate folders = independent work [OK]

Hint: Think about who edits which folder [OK]

Common Mistakes:

• Assuming splitting files speeds up project execution
• Believing all team members must edit same files
• Thinking staging models are hidden

4. A dbt team notices frequent merge conflicts because everyone edits models in the same folder. What is the best fix?

medium

A. Delete unused models to reduce files.

B. Ask everyone to edit only one model file.

C. Move models into separate folders by feature or domain.

D. Use random file names to confuse conflicts.

Solution

1. Step 1: Identify cause of conflicts

   Editing many models in one folder causes overlapping work and merge conflicts.

2. Step 2: Apply project structure best practice

   Separating models by feature or domain reduces overlap and conflicts.

3. Final Answer:

   Move models into separate folders by feature or domain. -> Option C

4. Quick Check:

   Separate folders reduce conflicts [OK]

Hint: Separate work areas to avoid editing clashes [OK]

Common Mistakes:

• Thinking deleting files solves conflicts
• Believing random names help avoid conflicts
• Assuming one file for all is better

5. Your dbt team has grown to 10 members working on different business areas. How should you organize the project structure to scale efficiently?

hard

A. Keep all models in one folder but use prefixes in file names.

B. Create separate folders for each business area and assign team members accordingly.

C. Let each member create their own folder with no naming rules.

D. Merge all models into a single large SQL file to simplify.

Solution

1. Step 1: Consider team size and work areas

   With many members, organizing by business area reduces overlap and confusion.

2. Step 2: Evaluate options for scalability

   Separate folders with clear ownership help manage changes and onboarding.

3. Step 3: Avoid poor practices

   Single folder or random folders cause conflicts; one big file is unmanageable.

4. Final Answer:

   Create separate folders for each business area and assign team members accordingly. -> Option B

5. Quick Check:

   Folder per area = scalable teamwork [OK]

Hint: Group by business area for clear ownership [OK]

Common Mistakes:

• Using one folder with prefixes causes conflicts
• Random folders confuse team members
• One big file is hard to maintain