Creating your own dbt package - Visual Walkthrough

Concept Flow - Creating your own dbt package

Start: Create package directory

↓

Add dbt_project.yml file

↓

Create models/ directory with SQL files

↓

Define package dependencies (optional)

↓

Run dbt commands: dbt deps, dbt build

↓

Package ready to use or share

This flow shows the steps to create a dbt package: set up folder, add config, add models, manage dependencies, then build.

Execution Sample

dbt

mkdir my_dbt_package
cd my_dbt_package
# create dbt_project.yml
# add models/my_model.sql
# run dbt build

This code sets up a new dbt package folder, adds config and model, then builds the package.

Execution Table

Step	Action	File/Folder Created	Command Run	Result
1	Create package folder	my_dbt_package/		Folder created
2	Add dbt_project.yml	my_dbt_package/dbt_project.yml		Project config added
3	Add models folder	my_dbt_package/models/		Models folder created
4	Add SQL model file	my_dbt_package/models/my_model.sql		Model SQL file added
5	Install dependencies		dbt deps	Dependencies installed (if any)
6	Build package		dbt build	Models compiled and run
7	Package ready			Package can be used or shared

💡 All steps completed, package created and built successfully

Variable Tracker

Variable	Start	After Step 2	After Step 4	After Step 6	Final
package_folder	none	my_dbt_package/	my_dbt_package/	my_dbt_package/	my_dbt_package/
dbt_project.yml	none	created	created	created	created
models_folder	none	none	created	created	created
model_sql_file	none	none	added	added	added
dependencies	none	none	none	installed	installed
build_status	none	none	none	success	success

Key Moments - 3 Insights

Why do we need the dbt_project.yml file?

The dbt_project.yml file tells dbt about your package settings and where to find models. Without it, dbt cannot build your package. See execution_table step 2.

What happens if you forget to add models/ folder?

Without the models folder and SQL files, dbt has nothing to build. The package will be empty. See execution_table step 3 and 4.

Why run 'dbt deps' before 'dbt build'?

'dbt deps' installs any package dependencies your package needs. Running it ensures all required packages are ready before building. See execution_table step 5.

Visual Quiz - 3 Questions

Look at the execution table, what is created at step 4?

A dbt_project.yml file

B models folder

C SQL model file

D package folder

💡 Hint

Check the 'File/Folder Created' column at step 4 in the execution_table

At which step does the package become ready to use?

A Step 6

B Step 7

C Step 5

D Step 4

💡 Hint

Look at the 'Result' column for the final step in the execution_table

If you skip 'dbt deps', what might happen?

A Dependencies might be missing

B Models won't compile

C Package folder won't be created

D dbt_project.yml will be ignored

💡 Hint

Refer to the explanation in key_moments about step 5 in execution_table

Concept Snapshot

Creating your own dbt package:
1. Make a new folder for your package.
2. Add a dbt_project.yml file to configure it.
3. Create a models/ folder and add SQL files.
4. Run 'dbt deps' to install dependencies.
5. Run 'dbt build' to compile and run models.
6. Package is ready to use or share.

Full Transcript

To create your own dbt package, start by making a new folder. Inside, add a dbt_project.yml file which tells dbt about your package settings. Then create a models folder and add SQL model files there. If your package depends on others, run 'dbt deps' to install them. Finally, run 'dbt build' to compile and run your models. After these steps, your package is ready to use or share with others.

Practice

1. What is the main purpose of creating your own dbt package?

easy

A. To reuse and share data models easily across projects

B. To write SQL queries faster

C. To create dashboards automatically

D. To store raw data files

Solution

1. Step 1: Understand the role of dbt packages

   dbt packages are designed to bundle data models and logic for reuse.

2. Step 2: Identify the main benefit

   Sharing and reusing models saves time and keeps projects consistent.

3. Final Answer:

   To reuse and share data models easily across projects -> Option A

4. Quick Check:

   dbt package purpose = reuse/share models [OK]

Hint: Think: Why share code? To reuse it easily [OK]

Common Mistakes:

• Confusing packages with dashboards
• Thinking packages store raw data
• Believing packages speed up SQL writing only

2. Which file do you edit to add a new dbt package to your main project?

easy

A. models.sql

B. dbt_project.yml

C. profiles.yml

D. packages.yml

Solution

1. Step 1: Recall where packages are declared

   dbt uses packages.yml to list external or local packages.

2. Step 2: Differentiate from other config files

   dbt_project.yml configures the project itself, not packages.

3. Final Answer:

   packages.yml -> Option D

4. Quick Check:

   Package list file = packages.yml [OK]

Hint: Add packages in packages.yml, not dbt_project.yml [OK]

Common Mistakes:

• Adding packages in dbt_project.yml
• Confusing profiles.yml with packages.yml
• Trying to add packages in SQL files

3. Given this packages.yml snippet:

packages:
  - local: ../my_dbt_package

What command installs this package into your project?

easy

A. dbt install

B. dbt build

C. dbt deps

D. dbt run

Solution

1. Step 1: Identify the command for package installation

   dbt uses dbt deps to download and install packages listed in packages.yml.

2. Step 2: Differentiate from other commands

   dbt run runs models, dbt install is not a valid command.

3. Final Answer:

   dbt deps -> Option C

4. Quick Check:

   Install packages with dbt deps [OK]

Hint: Use dbt deps to install packages listed in packages.yml [OK]

Common Mistakes:

• Using dbt install (not a valid command)
• Confusing dbt run with package install
• Trying dbt build instead of dbt deps

4. You created a package but when running dbt deps, you get an error: "Could not find package at path '../my_package'". What is the likely cause?

medium

A. The relative path in packages.yml is incorrect or folder missing

B. You forgot to run dbt run first

C. The package name is missing in dbt_project.yml

D. You need to delete the profiles.yml file

Solution

1. Step 1: Understand the error message

   "Could not find package at path" means dbt cannot locate the folder specified.

2. Step 2: Check the path in packages.yml

   Relative paths must be correct and folder must exist for local packages.

3. Final Answer:

   The relative path in packages.yml is incorrect or folder missing -> Option A

4. Quick Check:

   Path error = wrong folder path [OK]

Hint: Check folder path in packages.yml matches actual location [OK]

Common Mistakes:

• Running dbt run before dbt deps
• Editing dbt_project.yml instead of packages.yml
• Deleting unrelated config files

5. You want to use a model named customer_summary from your custom package my_package. Which is the correct way to reference it in your SQL model?

hard

A. select * from {{ ref('my_package', 'customer_summary') }}

B. select * from {{ ref('my_package.customer_summary') }}

C. select * from {{ ref('customer_summary') }}

D. select * from {{ ref('my_package__customer_summary') }}

Solution

1. Step 1: Recall how to reference models from packages

   Use ref('package_name.model_name') to refer to models in packages.

2. Step 2: Check each option's syntax

   select * from {{ ref('my_package.customer_summary') }} uses the correct single-argument ref syntax with dot notation for packages.

3. Final Answer:

   select * from {{ ref('my_package.customer_summary') }} -> Option B

4. Quick Check:

   Package model ref = ref('package.model') [OK]

Hint: Use ref('package.model') to access package models [OK]

Common Mistakes:

• Using ref('package', 'model') syntax (that's for sources)
• Forgetting the package name in ref
• Using double underscore instead of dot