Version pinning and updates in dbt - Mini Project: Build & Apply

Version pinning and updates

📖 Scenario: You are working on a data project using dbt. To keep your project stable, you want to pin the dbt version your project uses. Later, you will learn how to update this version safely.

🎯 Goal: Learn how to pin a specific dbt version in your packages.yml file and update it when needed.

📋 What You'll Learn

Create a packages.yml file with a pinned dbt version

Add a variable to hold the new version number

Update the pinned version using the variable

Print the updated packages.yml content

💡 Why This Matters

🌍 Real World

In real data projects, pinning dbt versions helps keep your transformations stable and reproducible.

💼 Career

Data engineers and analysts often manage dbt projects and need to control package versions to avoid unexpected errors.

1

Create the initial packages.yml with pinned dbt version

Create a string variable called packages_yml that contains the exact text:
packages: - package: dbt-labs/dbt_utils version: 0.8.0

dbt

# Create a string variable called packages_yml with the exact content
# Your code here

Hint

Use triple quotes or escaped newlines to create the multi-line string exactly as shown.

2

Add a variable for the new dbt version

Create a variable called new_version and set it to the string '0.9.1'.

dbt

packages_yml = "packages:\n  - package: dbt-labs/dbt_utils\n    version: 0.8.0"
# Create a variable new_version with value '0.9.1'
# Your code here

Hint

Just assign the string '0.9.1' to the variable new_version.

3

Update the pinned version in packages_yml using new_version

Replace the version number 0.8.0 in packages_yml with the value of new_version using the replace method. Assign the result back to packages_yml.

dbt

packages_yml = "packages:\n  - package: dbt-labs/dbt_utils\n    version: 0.8.0"
new_version = '0.9.1'
# Replace '0.8.0' with new_version in packages_yml
# Your code here

Hint

Use the string method replace to swap the old version with the new one.

4

Print the updated packages_yml content

Write a print statement to display the updated packages_yml string.

dbt

packages_yml = "packages:\n  - package: dbt-labs/dbt_utils\n    version: 0.8.0"
new_version = '0.9.1'
packages_yml = packages_yml.replace('0.8.0', new_version)
# Print the updated packages_yml
# Your code here

Hint

Use print(packages_yml) to show the updated content.

Practice

1. What is the main purpose of version pinning in a dbt project?

easy

A. To keep the project stable by fixing package or dbt core versions

B. To always use the latest version of dbt without restrictions

C. To automatically update all dependencies daily

D. To remove all version constraints from packages

Solution

1. Step 1: Understand version pinning concept

   Version pinning means fixing the version of packages or dbt core to avoid unexpected changes.

2. Step 2: Identify the purpose in dbt projects

   This helps keep the project stable and predictable by avoiding automatic upgrades that might break things.

3. Final Answer:

   To keep the project stable by fixing package or dbt core versions -> Option A

4. Quick Check:

   Version pinning = stability [OK]

Hint: Version pinning means fixing versions to avoid surprises [OK]

Common Mistakes:

• Thinking version pinning means always using latest versions
• Assuming it automatically updates packages
• Confusing pinning with removing constraints

2. Which of the following is the correct syntax to pin a dbt package version to exactly 1.0.0 in packages.yml?

easy

A. - package: dbt-labs/dbt_utils
version: ">=1.0.0,<2.0.0"

B. - package: dbt-labs/dbt_utils
version: "1.0.0"

C. - package: dbt-labs/dbt_utils
version: "=1.0.0"

D. - package: dbt-labs/dbt_utils
version: "~>1.0.0"

Solution

1. Step 1: Review version pinning syntax in packages.yml

   To pin exactly one version, use the version string with the exact version number in quotes, like "1.0.0".

2. Step 2: Analyze options

   - package: dbt-labs/dbt_utils version: ">=1.0.0,<2.0.0" uses a range, not exact pinning. - package: dbt-labs/dbt_utils version: "=1.0.0" uses an invalid syntax with '='. - package: dbt-labs/dbt_utils version: "~>1.0.0" uses a pessimistic operator (~>), which allows patch updates.

3. Final Answer:

   - package: dbt-labs/dbt_utils version: "1.0.0" -> Option B

4. Quick Check:

   Exact version pin = "1.0.0" [OK]

Hint: Exact pin uses just the version string in quotes [OK]

Common Mistakes:

• Using '=' before version number incorrectly
• Confusing range syntax with exact pin
• Using pessimistic operator when exact pin needed

3. Given the following packages.yml snippet, what version of dbt_utils will be installed if versions 0.8.0, 0.9.0, and 1.0.0 are available?

- package: dbt-labs/dbt_utils
  version: ">=0.8.0,<1.0.0"

medium

A. 0.9.0

B. 1.0.0

C. 0.8.0

D. Any version above 1.0.0

Solution

1. Step 1: Understand version range syntax

   The version range ">=0.8.0,<1.0.0" means any version starting from 0.8.0 up to but not including 1.0.0.

2. Step 2: Identify highest version in range

   Among available versions 0.8.0, 0.9.0, and 1.0.0, only 0.8.0 and 0.9.0 fit the range. The highest is 0.9.0.

3. Final Answer:

   0.9.0 -> Option A

4. Quick Check:

   Version <1.0.0 and >=0.8.0 = 0.9.0 [OK]

Hint: Pick highest version within the range [OK]

Common Mistakes:

• Choosing 1.0.0 which is excluded by <1.0.0
• Picking lowest version instead of highest
• Ignoring the upper bound of the range

4. You updated your packages.yml to use version: ">=0.8.0,<1.0.0" but dbt still installs version 1.0.0. What is the most likely cause?

medium

A. The version range syntax is incorrect and ignored

B. dbt always installs the latest version regardless of pinning

C. You forgot to run dbt deps after changing the file

D. The package name is misspelled

Solution

1. Step 1: Check version range syntax

   The syntax ">=0.8.0,<1.0.0" is correct and valid for version pinning.

2. Step 2: Consider dbt workflow

   After changing packages.yml, you must run dbt deps to update installed packages. Without this, old versions remain.

3. Final Answer:

   You forgot to run dbt deps after changing the file -> Option C

4. Quick Check:

   Run dbt deps after changes [OK]

Hint: Always run dbt deps after editing packages.yml [OK]

Common Mistakes:

• Assuming dbt auto-updates packages without command
• Thinking version range syntax is wrong
• Ignoring package installation step

5. You want to allow patch updates but prevent minor or major upgrades for the dbt_utils package. Which version pinning strategy in packages.yml achieves this while balancing stability and updates?

hard

A. - package: dbt-labs/dbt_utils
version: ">=1.0.0"

B. - package: dbt-labs/dbt_utils
version: ">=1.0.0,<2.0.0"

C. - package: dbt-labs/dbt_utils
version: "1.0.0"

D. - package: dbt-labs/dbt_utils
version: "~>1.0.0"

Solution

1. Step 1: Understand version pinning operators

   Exact version ("1.0.0") disallows all updates. Range (">=1.0.0,<2.0.0") allows minor updates (1.x) but blocks major. ">=1.0.0" allows minor and major upgrades with no upper bound.

2. Step 2: Identify the pessimistic operator usage

   The "~>1.0.0" operator allows patch updates (like 1.0.1, 1.0.2) but prevents minor (1.1.0) or major (2.0.0) upgrades, balancing stability and updates.

3. Final Answer:

   - package: dbt-labs/dbt_utils version: "~>1.0.0" -> Option D

4. Quick Check:

   Pessimistic operator (~>) allows patch updates only [OK]

Hint: Use ~> for patch updates only, blocking minor/major [OK]

Common Mistakes:

• Using exact version and missing updates
• Using wide range allowing minor/major updates
• Confusing range with pessimistic operator