Environment management (dev, staging, prod) in dbt - Step-by-Step Execution

Concept Flow - Environment management (dev, staging, prod)

Start: Choose Environment

↓

Use dev settings

No↓

Use staging settings

No↓

Use prod settings

↓

Run dbt commands with chosen environment

↓

Deploy or test changes

↓

End

This flow shows how dbt selects settings based on environment (dev, staging, prod) and runs commands accordingly.

Execution Sample

dbt

profiles:
  my_project:
    target: dev
    outputs:
      dev:
        type: snowflake
        ...
      staging:
        type: snowflake
        ...
      prod:
        type: snowflake
        ...

This dbt profile configures three environments with different connection settings.

Execution Table

Step	Environment Variable	Target Selected	Action	Result
1	DBT_ENV=dev	dev	Load dev settings	Connect to dev database
2	Run dbt run	dev	Compile and run models	Models built in dev schema
3	DBT_ENV=staging	staging	Load staging settings	Connect to staging database
4	Run dbt test	staging	Run tests on staging	Tests executed on staging schema
5	DBT_ENV=prod	prod	Load prod settings	Connect to production database
6	Run dbt run	prod	Compile and run models	Models built in prod schema
7	DBT_ENV=unknown	prod (default)	Load prod settings	Connect to production database
8	Run dbt run	prod	Compile and run models	Models built in prod schema

💡 Execution stops after running commands in the selected environment.

Variable Tracker

Variable	Start	After Step 1	After Step 3	After Step 5	After Step 7
DBT_ENV	undefined	dev	staging	prod	unknown
target	undefined	dev	staging	prod	prod (default)
connection	undefined	dev connection	staging connection	prod connection	prod connection

Key Moments - 3 Insights

Why does dbt use the prod environment when DBT_ENV is unknown?

dbt defaults to the 'prod' target if the environment variable is not set or unknown, as shown in execution_table row 7.

How does dbt know which database to connect to?

dbt reads the target environment from DBT_ENV and loads the matching profile settings, as seen in steps 1, 3, and 5.

Can you run tests in the dev environment?

Yes, you can run tests in any environment by setting DBT_ENV accordingly and running 'dbt test', similar to step 4 for staging.

Visual Quiz - 3 Questions

Look at the execution_table at step 3. What environment is selected?

A staging

B prod

C dev

D unknown

💡 Hint

Check the 'Target Selected' column at step 3 in the execution_table.

At which step does dbt run models in the production environment?

A Step 2

B Step 6

C Step 4

D Step 8

💡 Hint

Look for 'prod' in 'Target Selected' and 'Run dbt run' in 'Action' column.

If DBT_ENV is set to 'dev', what will be the connection after step 1 according to variable_tracker?

A staging connection

B prod connection

C dev connection

D undefined

💡 Hint

Check 'connection' variable value after step 1 in variable_tracker.

Concept Snapshot

Environment management in dbt:
- Use DBT_ENV variable to select environment
- Profiles.yml defines dev, staging, prod targets
- dbt commands run with selected target
- Defaults to prod if unknown
- Enables safe testing and deployment

Full Transcript

This visual execution shows how dbt manages environments like dev, staging, and prod. It starts by reading the DBT_ENV variable to select the target environment. Based on this, dbt loads the corresponding connection settings from profiles.yml. Then, dbt runs commands like 'dbt run' or 'dbt test' in the chosen environment. If DBT_ENV is not set or unknown, dbt defaults to the production environment to avoid accidental use of wrong settings. Variables like DBT_ENV, target, and connection update step-by-step as commands execute. This helps keep development, testing, and production separate and safe.