Test severity levels in dbt - Step-by-Step Execution

Concept Flow - Test severity levels

Define test in dbt

↓

Assign severity level

↓

Run dbt tests

↓

Test result: Pass or Fail

↓

If Fail -> Check severity

↓

Fail build

↓

Report results

This flow shows how dbt tests are defined with severity levels, run, and how failures are handled differently based on severity.

Execution Sample

dbt

version: 2
models:
  - name: customers
    tests:
      - unique:
          column_name: id
        config:
          severity: error
      - not_null:
          column_name: id
        config:
          severity: warn

Defines two tests on the customers model with different severity levels: error and warn.

Execution Table

Step	Test Name	Severity	Test Result	Action Taken	Build Status
1	unique	error	pass	No action	continue
2	not_null	warn	fail	Log warning	continue
3	unique	error	fail	Fail build	stop
4	not_null	warn	pass	No action	stop
5	unique	error	pass	No action	stop
6	not_null	warn	fail	Log warning	stop

💡 Build stops immediately if any test with severity 'error' fails; warnings do not stop the build.

Variable Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	After Step 6
Build Status	continue	continue	continue	stop	stop	stop	stop
Warnings Logged	0	0	1	1	1	1	2

Key Moments - 2 Insights

Why does the build stop at step 3 but not at step 2 even though both tests fail?

Because step 3 fails a test with severity 'error' which stops the build immediately, while step 2 fails a 'warn' severity test that only logs a warning and continues. See execution_table rows 2 and 3.

What happens if multiple warn severity tests fail?

All warn failures log warnings but do not stop the build. The build continues running all tests. See execution_table rows 2 and 6 and variable_tracker for warnings count.

Visual Quiz - 3 Questions

Look at the execution table, what is the build status after step 2?

A stop

B continue

C paused

D unknown

💡 Hint

Check the 'Build Status' column at step 2 in the execution_table.

At which step does the build stop due to a test failure?

A Step 3

B Step 2

C Step 6

D Step 1

💡 Hint

Look for the first 'stop' in the 'Build Status' column in the execution_table.

If the severity of the unique test was changed from 'error' to 'warn', what would happen at step 3?

A Build would stop

B Test would pass automatically

C Build would continue with a warning

D Test would be skipped

💡 Hint

Refer to how 'warn' severity tests behave in the execution_table and key_moments.

Concept Snapshot

dbt test severity levels:
- severity: error stops the build on failure
- severity: warn logs a warning but continues
- define severity in test config
- errors are critical, warnings are advisory
- build status depends on test severity results

Full Transcript

In dbt, tests can have severity levels like 'error' or 'warn'. When tests run, if an 'error' severity test fails, the build stops immediately. If a 'warn' severity test fails, it logs a warning but the build continues. This allows teams to prioritize critical issues while still being aware of less critical ones. The execution table shows test results step-by-step, tracking build status and warnings logged. Understanding severity helps manage test failures effectively.

Practice

1. What does setting a dbt test severity level to ERROR do?

easy

A. It stops the dbt run if the test fails.

B. It only logs a warning but continues the run.

C. It ignores the test result completely.

D. It retries the test automatically.

Solution

1. Step 1: Understand dbt test severity levels

   dbt uses severity levels to decide what happens when a test fails.

2. Step 2: Identify the effect of ERROR severity

   When severity is set to ERROR, dbt stops the run immediately on failure.

3. Final Answer:

   It stops the dbt run if the test fails. -> Option A

4. Quick Check:

   ERROR severity = stops run [OK]

Hint: ERROR severity stops the run; WARN just warns [OK]

Common Mistakes:

• Confusing ERROR with WARN severity
• Thinking ERROR retries the test
• Assuming ERROR ignores failures

2. Which of the following is the correct way to set a test severity to WARN in a dbt YAML test configuration?

easy

A. severity: warn

B. severity = WARN

C. severity: WARN

D. severity: Warning

Solution

1. Step 1: Recall YAML syntax for dbt test severity

   dbt expects severity as a key-value pair with uppercase values like WARN or ERROR.

2. Step 2: Identify correct syntax

   The correct syntax uses a colon and uppercase WARN: severity: WARN.

3. Final Answer:

   severity: WARN -> Option C

4. Quick Check:

   YAML key-value with uppercase WARN = correct [OK]

Hint: Use colon and uppercase WARN for severity [OK]

Common Mistakes:

• Using lowercase 'warn' instead of uppercase
• Using equals sign instead of colon
• Spelling severity value incorrectly

3. Given this dbt test configuration snippet:

tests:
  - unique:
      column_name: id
      severity: WARN

What happens if the test fails during a dbt run?

medium

A. The failure is logged as a warning, but the run continues.

B. The test automatically retries until it passes.

C. The test is ignored and no message is shown.

D. The run stops immediately with an error.

Solution

1. Step 1: Analyze the severity level in the test config

   The severity is set to WARN, which means dbt should warn but not stop.

2. Step 2: Understand dbt behavior on WARN severity

   When a test fails with WARN severity, dbt logs a warning and continues the run.

3. Final Answer:

   The failure is logged as a warning, but the run continues. -> Option A

4. Quick Check:

   WARN severity = warn and continue [OK]

Hint: WARN severity warns but lets run continue [OK]

Common Mistakes:

• Thinking WARN stops the run
• Assuming WARN ignores failures
• Believing tests retry automatically

4. You wrote this test in your dbt model YAML:

tests:
  - not_null:
      column_name: user_id
      severity: ERROR

But your dbt run does not stop when the test fails. What is the likely problem?

medium

A. The test name 'not_null' is invalid.

B. The severity key is misplaced; it should be under 'config'.

C. The severity value should be lowercase 'error'.

D. The test is not actually failing.

Solution

1. Step 1: Check correct placement of severity in dbt tests

   Severity must be set inside the config block, not directly under the test.

2. Step 2: Identify why run does not stop

   Since severity is misplaced, dbt ignores it and uses default behavior, so run does not stop.

3. Final Answer:

   The severity key is misplaced; it should be under 'config'. -> Option B

4. Quick Check:

   Severity must be inside config = The severity key is misplaced; it should be under 'config'. [OK]

Hint: Put severity inside config block to take effect [OK]

Common Mistakes:

• Using lowercase severity value
• Misplacing severity outside config
• Assuming test name is wrong

5. You want to run a dbt test that warns on missing emails but errors on duplicate emails. Which YAML configuration correctly sets different severities for these tests on the email column?

hard

A. tests:
- not_null:
column_name: email
severity: WARN
- unique:
column_name: email
severity: ERROR

B. tests:
- not_null:
column_name: email
config:
severity: ERROR
- unique:
column_name: email
config:
severity: WARN

C. tests:
- not_null:
column_name: email
severity: ERROR
- unique:
column_name: email
severity: WARN

D. tests:
- not_null:
column_name: email
config:
severity: WARN
- unique:
column_name: email
config:
severity: ERROR

Solution

1. Step 1: Recall correct severity placement in dbt YAML

   Severity must be inside a config block under each test.

2. Step 2: Match severities to tests as required

   Set not_null severity to WARN and unique severity to ERROR inside their config blocks.

3. Final Answer:

   Tests with severity inside config: not_null WARN, unique ERROR. -> Option D

4. Quick Check:

   Severity inside config with correct WARN/ERROR = tests: - not_null: column_name: email config: severity: WARN - unique: column_name: email config: severity: ERROR [OK]

Hint: Use config block for different severities per test [OK]

Common Mistakes:

• Placing severity outside config block
• Swapping WARN and ERROR severities
• Using lowercase severity values