Environment management (dev, staging, prod) in dbt

Introduction

Environment management helps keep your work organized and safe. It separates your development, testing, and live data so you don't mix them up.

When you want to try new changes without affecting live data.

When you need to test your data models before sharing results.

When you want to review changes with your team before going live.

When you want to keep your production data safe from errors.

When you want to deploy updates step-by-step.

Syntax

dbt

profiles.yml

my_profile:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      user: user_dev
      password: pass_dev
      dbname: mydb_dev
      schema: dev_schema
    staging:
      type: postgres
      host: staging_host
      user: user_staging
      password: pass_staging
      dbname: mydb_staging
      schema: staging_schema
    prod:
      type: postgres
      host: prod_host
      user: user_prod
      password: pass_prod
      dbname: mydb_prod
      schema: prod_schema

The profiles.yml file defines connection details for each environment.

You switch environments by changing the target in your profile or using the --target flag when running dbt commands.

Examples

This runs your dbt models in the development environment.

dbt

dbt run --target dev

This runs your dbt models in the staging environment for testing.

dbt

dbt run --target staging

This runs your dbt models in the production environment, updating live data.

dbt

dbt run --target prod

Sample Program

This example shows how to run dbt commands in different environments using the --target flag. Each environment uses its own database and schema to keep data separate.

dbt

# profiles.yml example

# Define three environments: dev, staging, prod

# Run dbt in dev environment
!dbt run --target dev

# Run dbt in staging environment
!dbt run --target staging

# Run dbt in prod environment
!dbt run --target prod

Important Notes

Always test your changes in dev or staging before running in prod.

Keep your credentials safe and do not share production passwords.

You can automate environment switching in CI/CD pipelines for smooth deployments.

Summary

Environment management separates your work into dev, staging, and prod.

Use profiles.yml to set connection details for each environment.

Switch environments easily with the --target flag when running dbt commands.