Creating your own dbt package - Why You Should Know This

The Big Idea

What if fixing one bug could fix it everywhere instantly?

The Scenario

Imagine you have many data projects, and each needs similar data transformations. You copy and paste code between projects. When you find a mistake, you fix it in one place but forget others. It feels like juggling many balls, and some always drop.

The Problem

Manually copying code is slow and risky. You waste time fixing the same bug multiple times. It's easy to lose track of changes. Your projects become messy and hard to maintain. Collaboration is tough because everyone works on different copies.

The Solution

Creating your own dbt package lets you write your data transformation code once. Then you share it across projects easily. Updates happen in one place and flow everywhere. It keeps your work clean, consistent, and easy to manage.

Before vs After

✗ Before

copy code from project A to B
fix bug in A
forget to fix in B

✓ After

create dbt package
install package in projects
update package once

What It Enables

You can build reusable, reliable data transformations that save time and reduce errors across all your projects.

Real Life Example

A company has multiple teams building reports. By creating a shared dbt package for common data cleaning, all teams get consistent, up-to-date data without repeating work.

Key Takeaways

Manual code copying causes errors and wastes time.

dbt packages let you reuse and share code easily.

One update fixes all projects using the package.

Practice

1. What is the main purpose of creating your own dbt package?

easy

A. To reuse and share data models easily across projects

B. To write SQL queries faster

C. To create dashboards automatically

D. To store raw data files

Solution

1. Step 1: Understand the role of dbt packages

   dbt packages are designed to bundle data models and logic for reuse.

2. Step 2: Identify the main benefit

   Sharing and reusing models saves time and keeps projects consistent.

3. Final Answer:

   To reuse and share data models easily across projects -> Option A

4. Quick Check:

   dbt package purpose = reuse/share models [OK]

Hint: Think: Why share code? To reuse it easily [OK]

Common Mistakes:

• Confusing packages with dashboards
• Thinking packages store raw data
• Believing packages speed up SQL writing only

2. Which file do you edit to add a new dbt package to your main project?

easy

A. models.sql

B. dbt_project.yml

C. profiles.yml

D. packages.yml

Solution

1. Step 1: Recall where packages are declared

   dbt uses packages.yml to list external or local packages.

2. Step 2: Differentiate from other config files

   dbt_project.yml configures the project itself, not packages.

3. Final Answer:

   packages.yml -> Option D

4. Quick Check:

   Package list file = packages.yml [OK]

Hint: Add packages in packages.yml, not dbt_project.yml [OK]

Common Mistakes:

• Adding packages in dbt_project.yml
• Confusing profiles.yml with packages.yml
• Trying to add packages in SQL files

3. Given this packages.yml snippet:

packages:
  - local: ../my_dbt_package

What command installs this package into your project?

easy

A. dbt install

B. dbt build

C. dbt deps

D. dbt run

Solution

1. Step 1: Identify the command for package installation

   dbt uses dbt deps to download and install packages listed in packages.yml.

2. Step 2: Differentiate from other commands

   dbt run runs models, dbt install is not a valid command.

3. Final Answer:

   dbt deps -> Option C

4. Quick Check:

   Install packages with dbt deps [OK]

Hint: Use dbt deps to install packages listed in packages.yml [OK]

Common Mistakes:

• Using dbt install (not a valid command)
• Confusing dbt run with package install
• Trying dbt build instead of dbt deps

4. You created a package but when running dbt deps, you get an error: "Could not find package at path '../my_package'". What is the likely cause?

medium

A. The relative path in packages.yml is incorrect or folder missing

B. You forgot to run dbt run first

C. The package name is missing in dbt_project.yml

D. You need to delete the profiles.yml file

Solution

1. Step 1: Understand the error message

   "Could not find package at path" means dbt cannot locate the folder specified.

2. Step 2: Check the path in packages.yml

   Relative paths must be correct and folder must exist for local packages.

3. Final Answer:

   The relative path in packages.yml is incorrect or folder missing -> Option A

4. Quick Check:

   Path error = wrong folder path [OK]

Hint: Check folder path in packages.yml matches actual location [OK]

Common Mistakes:

• Running dbt run before dbt deps
• Editing dbt_project.yml instead of packages.yml
• Deleting unrelated config files

5. You want to use a model named customer_summary from your custom package my_package. Which is the correct way to reference it in your SQL model?

hard

A. select * from {{ ref('my_package', 'customer_summary') }}

B. select * from {{ ref('my_package.customer_summary') }}

C. select * from {{ ref('customer_summary') }}

D. select * from {{ ref('my_package__customer_summary') }}

Solution

1. Step 1: Recall how to reference models from packages

   Use ref('package_name.model_name') to refer to models in packages.

2. Step 2: Check each option's syntax

   select * from {{ ref('my_package.customer_summary') }} uses the correct single-argument ref syntax with dot notation for packages.

3. Final Answer:

   select * from {{ ref('my_package.customer_summary') }} -> Option B

4. Quick Check:

   Package model ref = ref('package.model') [OK]

Hint: Use ref('package.model') to access package models [OK]

Common Mistakes:

• Using ref('package', 'model') syntax (that's for sources)
• Forgetting the package name in ref
• Using double underscore instead of dot