Installing packages with packages.yml in dbt - Visual Walkthrough

Concept Flow - Installing packages with packages.yml

Create packages.yml file

↓

Add package details

↓

Run 'dbt deps' command

↓

dbt downloads packages

↓

Packages installed in 'dbt_packages'

↓

Use packages in your project

This flow shows how to add package info to packages.yml, run dbt deps to download, and then use the packages.

Execution Sample

dbt

packages:
  - package: fishtown-analytics/dbt_utils
    version: 0.8.6

This packages.yml file tells dbt to install the dbt_utils package version 0.8.6.

Execution Table

Step	Action	Command/Content	Result
1	Create packages.yml	Add package name and version	File created with package info
2	Run dbt deps	dbt deps	dbt reads packages.yml
3	Download packages	dbt downloads dbt_utils v0.8.6	Package saved in dbt_packages
4	Verify installation	Check dbt_packages folder	dbt_utils package present
5	Use package	Reference dbt_utils in models	Package functions available
6	Exit	No more packages to install	Process complete

💡 All packages listed in packages.yml are installed and ready to use.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	Final
packages.yml content	empty	package: fishtown-analytics/dbt_utils v0.8.6	read by dbt deps	package downloaded	package ready in dbt_packages
dbt_packages folder	empty	empty	empty	dbt_utils v0.8.6 present	dbt_utils usable

Key Moments - 2 Insights

Why do we need to run 'dbt deps' after creating packages.yml?

Because dbt reads packages.yml only when you run 'dbt deps' to download and install the packages, as shown in steps 2 and 3 of the execution_table.

What happens if the version number in packages.yml is missing or incorrect?

dbt may download the latest version or fail to find the package. The execution_table step 3 shows the importance of specifying the correct version.

Visual Quiz - 3 Questions

Look at the execution table, what is the result after running 'dbt deps'?

A dbt reads packages.yml

B Package functions become available

C File packages.yml is created

D dbt_packages folder is deleted

💡 Hint

Check Step 2 in the execution_table where 'dbt deps' is run.

At which step does the package get downloaded into dbt_packages?

A Step 1

B Step 5

C Step 3

D Step 6

💡 Hint

Look at Step 3 in the execution_table for the download action.

If you forget to run 'dbt deps', what will happen when you try to use the package?

A dbt will automatically download the package

B The package will not be found and cause errors

C The package functions will work normally

D The packages.yml file will be deleted

💡 Hint

Refer to the key_moments about running 'dbt deps' and the execution_table steps.

Concept Snapshot

packages.yml lists packages with name and version
Run 'dbt deps' to download packages
Packages saved in dbt_packages folder
Use package functions in your dbt models
Always specify correct version
Run 'dbt deps' after editing packages.yml

Full Transcript

To install packages in dbt, you create a packages.yml file listing the packages and their versions. Then you run the command 'dbt deps' which reads this file and downloads the packages into the dbt_packages folder. After this, you can use the package functions in your dbt project. If you do not run 'dbt deps', the packages will not be downloaded and your project will not find them. Specifying the correct version in packages.yml ensures you get the right package version.

Practice

1. What is the main purpose of the packages.yml file in a dbt project?

easy

A. To list external packages and their versions for installation

B. To define database connection settings

C. To write SQL models for transformations

D. To configure dbt user permissions

Solution

1. Step 1: Understand the role of packages.yml and compare with other dbt files

   This file is used to specify external packages that the dbt project depends on, including their names and versions. Database connections are set in profiles.yml, SQL models are separate files, and permissions are managed outside packages.yml.

2. Final Answer:

   To list external packages and their versions for installation -> Option A

3. Quick Check:

   packages.yml = package list [OK]

Hint: Remember: packages.yml lists packages, not configs or code [OK]

Common Mistakes:

• Confusing packages.yml with profiles.yml
• Thinking packages.yml contains SQL code
• Assuming it manages user permissions

2. Which of the following is the correct syntax to specify a package in packages.yml?

easy

A. - dbt_utils: 0.8.0

B. package: dbt_utils, version=0.8.0

C. dbt_utils: version 0.8.0

D. - package: dbt_utils
version: 0.8.0

Solution

1. Step 1: Recall YAML list syntax for packages and check each option format

   Packages are listed as items with a dash, followed by keys package and version indented below. - package: dbt_utils version: 0.8.0 matches the correct YAML format. Others use incorrect separators or structure.

2. Final Answer:

   - package: dbt_utils version: 0.8.0 -> Option D

3. Quick Check:

   YAML list with dash and keys = - package: dbt_utils version: 0.8.0 [OK]

Hint: YAML lists use dash and indentation for items [OK]

Common Mistakes:

• Using commas instead of new lines
• Missing dash for list items
• Incorrect key-value separators

3. Given this packages.yml content:

- package: dbt_utils
  version: 0.8.0
- package: fivetran_utils
  version: 0.3.0

What command should you run to install these packages?

medium

A. dbt deps

B. dbt install

C. dbt run

D. dbt build

Solution

1. Step 1: Identify the command for installing packages and differentiate from other dbt commands

   The command dbt deps reads packages.yml and installs or updates the listed packages. dbt install is not a valid command, dbt run runs models, and dbt build runs models and tests but does not install packages.

2. Final Answer:

   dbt deps -> Option A

3. Quick Check:

   Install packages = dbt deps [OK]

Hint: Use 'dbt deps' to install packages from packages.yml [OK]

Common Mistakes:

• Using 'dbt install' which doesn't exist
• Confusing 'dbt run' with package installation
• Trying 'dbt build' for package install

4. You added a new package to packages.yml but after running dbt deps, the package is not installed. What is the most likely reason?

medium

A. The package version is too new and unsupported

B. You ran dbt run instead of dbt deps

C. You forgot to save the packages.yml file before running the command

D. You need to restart your computer

Solution

1. Step 1: Check common user errors and evaluate other options

   If the packages.yml file was not saved, dbt deps reads the old file without the new package. Running dbt run does not install packages, but the question states dbt deps was run. Version issues usually cause errors, not silent skips. Restarting computer is unnecessary.

2. Final Answer:

   You forgot to save the packages.yml file before running the command -> Option C

3. Quick Check:

   Unsaved file = no package install [OK]

Hint: Always save packages.yml before running dbt deps [OK]

Common Mistakes:

• Assuming dbt run installs packages
• Ignoring file save before command
• Thinking restart fixes package install

5. You want to add the dbt_utils package version 0.8.0 and the fivetran_utils package version 0.3.0 to your project. Which packages.yml content is correct to install both packages properly?

hard

A. - dbt_utils: version 0.8.0
- fivetran_utils: version 0.3.0

B. - package: dbt_utils
version: 0.8.0
- package: fivetran_utils
version: 0.3.0

C. packages:
- dbt_utils: 0.8.0
- fivetran_utils: 0.3.0

D. - package: dbt_utils, version: 0.8.0
- package: fivetran_utils, version: 0.3.0

Solution

1. Step 1: Review correct YAML list format for multiple packages and check each option's correctness

   Each package is an item starting with a dash, followed by package and version keys on separate lines with indentation. - package: dbt_utils version: 0.8.0 - package: fivetran_utils version: 0.3.0 matches the correct YAML syntax. packages: - dbt_utils: 0.8.0 - fivetran_utils: 0.3.0 incorrectly adds a root key packages. - dbt_utils: version 0.8.0 - fivetran_utils: version 0.3.0 uses wrong key-value format. - package: dbt_utils, version: 0.8.0 - package: fivetran_utils, version: 0.3.0 uses commas inside YAML which is invalid.

2. Final Answer:

   - package: dbt_utils version: 0.8.0 - package: fivetran_utils version: 0.3.0 -> Option B

3. Quick Check:

   Proper YAML list with keys = - package: dbt_utils version: 0.8.0 - package: fivetran_utils version: 0.3.0 [OK]

Hint: List each package with dash, package, and version keys [OK]

Common Mistakes:

• Adding extra root keys like 'packages:'
• Using commas inside YAML list items
• Incorrect key-value formatting