Environment management (dev, staging, prod) in dbt - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of having separate environments like dev, staging, and prod in dbt?

Separate environments help keep work organized and safe.
Dev is for building and testing new changes.
Staging is for final testing before going live.
Prod is the live environment where real data is used.

beginner

How does dbt use profiles.yml to manage different environments?

The profiles.yml file stores connection details for each environment.
It tells dbt where to run models for dev, staging, or prod.
You switch environments by changing the target in dbt commands.

beginner

Why should you avoid running untested models directly in the prod environment?

Running untested models in prod can cause errors or bad data.
It can break reports or dashboards that people rely on.
Testing in dev and staging helps catch problems early.

intermediate

What is a common workflow for promoting changes from dev to prod in dbt?

1. Develop and test models in dev.
2. Deploy to staging for final checks.
3. After approval, deploy to prod.
This keeps data safe and reliable.

beginner

How can you use dbt commands to specify which environment to run?

Use the --target flag with dbt commands.
Example: dbt run --target staging runs models in the staging environment.
This tells dbt which profile settings to use.

What environment in dbt is typically used for final testing before production?

A Production

B Development

C Staging

D Testing

Staging is the environment used for final testing before deploying to production.

Which file in dbt holds the connection details for different environments?

A profiles.yml

B dbt_project.yml

C settings.yml

D environment.yml

The profiles.yml file contains connection details for dev, staging, and prod environments.

How do you tell dbt to run models in the production environment?

A dbt run --env prod

B dbt run --target prod

C dbt run --profile prod

D dbt run --environment production

The --target flag specifies the environment, so 'dbt run --target prod' runs models in production.

Why is it important to have a dev environment separate from prod?

A To save money on cloud costs

B To avoid using version control

C To speed up production queries

D To test changes without affecting live data

Dev environment allows testing without risking live production data.

What is the risk of skipping staging and deploying directly from dev to prod?

A Potential data errors and broken reports

B No risk at all

C Faster deployment

D Better collaboration

Skipping staging can cause errors in production and affect data quality.

Explain the roles of dev, staging, and prod environments in dbt and why each is important.

Think about how you would safely make changes to a recipe before serving it to guests.

Describe how you would use dbt commands and profiles.yml to switch between environments.

Imagine choosing which kitchen to cook in by telling dbt where to connect.

Practice

1. What is the main purpose of having separate environments like dev, staging, and prod in dbt?

easy

A. To run multiple dbt projects at the same time

B. To safely develop and test changes before applying them to production

C. To store different versions of dbt itself

D. To backup data automatically

Solution

1. Step 1: Understand environment roles

   Development (dev) is for building and testing changes. Staging is for final testing. Production (prod) is the live environment.

2. Step 2: Purpose of separation

   This separation prevents untested changes from affecting live data and users.

3. Final Answer:

   To safely develop and test changes before applying them to production -> Option B

4. Quick Check:

   Environment separation = safe testing before production [OK]

Hint: Think: Why not test directly in production? [OK]

Common Mistakes:

• Confusing environment roles
• Thinking environments store dbt versions
• Assuming environments backup data automatically

2. Which command correctly runs dbt models in the staging environment?

easy

A. dbt run --profile staging

B. dbt run --env staging

C. dbt run --target staging

D. dbt run --environment staging

Solution

1. Step 1: Identify the correct flag for environment selection

   dbt uses the --target flag to specify which environment to run against.

2. Step 2: Match the flag with the environment name

   Using --target staging runs models in the staging environment.

3. Final Answer:

   dbt run --target staging -> Option C

4. Quick Check:

   Use --target to switch environments [OK]

Hint: Remember: --target selects environment in dbt commands [OK]

Common Mistakes:

• Using --env or --environment instead of --target
• Confusing --profile with environment selection
• Omitting the flag and running default environment

3. Given the following profiles.yml snippet, what database will dbt connect to when running with --target prod?

my_project:
  outputs:
    dev:
      type: postgres
      host: dev-db.example.com
      dbname: dev_db
    prod:
      type: postgres
      host: prod-db.example.com
      dbname: prod_db
  target: dev

medium

A. prod_db on prod-db.example.com

B. dev_db on dev-db.example.com

C. dev_db on prod-db.example.com

D. prod_db on dev-db.example.com

Solution

1. Step 1: Understand the --target flag effect

   Running with --target prod tells dbt to use the prod output configuration.

2. Step 2: Identify prod output details

   The prod output connects to prod-db.example.com and database prod_db.

3. Final Answer:

   prod_db on prod-db.example.com -> Option A

4. Quick Check:

   --target prod uses prod output config [OK]

Hint: Match --target value to profiles.yml output key [OK]

Common Mistakes:

• Ignoring --target and using default target
• Mixing host and dbname from different environments
• Assuming target in profiles.yml overrides command line

4. You run dbt run --target staging but get a connection error. The profiles.yml has a typo in the staging host key as hst instead of host. What is the best fix?

medium

A. Run dbt clean before running again

B. Change the command to dbt run --target hst

C. Remove the staging environment from profiles.yml

D. Correct the typo from hst to host in profiles.yml

Solution

1. Step 1: Identify cause of connection error

   A typo in the key hst means dbt cannot find the host to connect to.

2. Step 2: Fix the typo in profiles.yml

   Changing hst to host corrects the connection details.

3. Final Answer:

   Correct the typo from hst to host in profiles.yml -> Option D

4. Quick Check:

   Fix config typos to resolve connection errors [OK]

Hint: Check spelling of keys in profiles.yml for connection issues [OK]

Common Mistakes:

• Changing command flags instead of fixing config
• Deleting environment instead of fixing it
• Running unrelated dbt commands

5. You want to test a new model in your dev environment without affecting staging or prod. Which approach is best to manage this safely?

hard

A. Create a separate dev target in profiles.yml and run with --target dev

B. Run the model directly in prod and rollback if needed

C. Delete the staging environment temporarily

D. Run all models in dev and prod simultaneously

Solution

1. Step 1: Understand environment isolation

   Using a separate dev target isolates testing from staging and production.

2. Step 2: Use --target dev to run safely

   Running with --target dev ensures changes only affect the dev environment.

3. Final Answer:

   Create a separate dev target in profiles.yml and run with --target dev -> Option A

4. Quick Check:

   Use dev target for safe isolated testing [OK]

Hint: Always test in dev environment using --target dev [OK]

Common Mistakes:

• Testing directly in production
• Removing staging environment
• Running multiple environments at once