Test severity levels in dbt - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding dbt test severity levels

In dbt, tests can have different severity levels that affect how failures are handled during runs. Which severity level causes dbt to fail the entire run if the test fails?

A pass

B error

C info

D warn

💡 Hint

Think about which severity level means the test failure is critical enough to stop the process.

The error severity level causes dbt to fail the entire run if the test fails. The warn level only logs a warning but continues the run.

❓ Predict Output

intermediate

Output of dbt test with warn severity

Given a dbt test configured with severity: warn and the test fails, what is the expected behavior in the dbt run output?

dbt

version: 2
models:
  - name: customers
    tests:
      - unique:
          column_name: id
          config:
            severity: warn

A The run logs a warning but continues to completion.

B The run fails and stops immediately.

C The run ignores the test and does not log anything.

D The run logs an error and retries the test.

💡 Hint

Severity 'warn' means the test failure is noted but does not stop the run.

When a test has severity 'warn' and fails, dbt logs a warning message but continues running other models and tests.

❓ data_output

advanced

Number of failed tests by severity

After running dbt tests, you get a summary table with test results and their severity levels. How many tests with severity 'error' failed if the table below is given?

test_name       | severity | status
unique_id       | error    | fail
not_null_email  | warn     | pass
unique_email    | error    | fail
accepted_values | warn     | fail

A 2

B 3

C 1

D 0

💡 Hint

Count only tests with severity 'error' and status 'fail'.

There are two tests with severity 'error' that failed: unique_id and unique_email.

🔧 Debug

advanced

Identify severity misconfiguration causing unexpected run failure

A dbt test is configured as follows but the run fails unexpectedly when the test fails. Identify the misconfiguration.

tests:
  - not_null:
      column_name: user_id
      severity: warn

However, the run fails on this test failure.

A Severity must be uppercase: <code>WARN</code> instead of <code>warn</code>.

B Severity 'warn' is not supported and causes failure.

C The test name is incorrect; it should be <code>not_null_test</code>.

D Severity should be set under <code>config</code> block, not directly under test.

💡 Hint

Check where severity is placed in the test configuration.

Severity must be set inside a config block for the test, not directly under the test name. Placing it incorrectly causes dbt to ignore it and default to 'error'.

🚀 Application

expert

Choosing severity levels for different test scenarios

You manage a dbt project with critical and non-critical data quality tests. Which severity level should you assign to tests that must never fail without stopping the pipeline, and which to tests that should only log issues but allow the pipeline to continue?

A Critical tests: warn; Non-critical tests: error

B Critical tests: pass; Non-critical tests: warn

C Critical tests: error; Non-critical tests: warn

D Critical tests: info; Non-critical tests: error

💡 Hint

Think about which severity stops the pipeline and which only logs warnings.

Critical tests should have severity 'error' to stop the pipeline on failure. Non-critical tests should have severity 'warn' to log issues but continue.

Practice

1. What does setting a dbt test severity level to ERROR do?

easy

A. It stops the dbt run if the test fails.

B. It only logs a warning but continues the run.

C. It ignores the test result completely.

D. It retries the test automatically.

Solution

1. Step 1: Understand dbt test severity levels

   dbt uses severity levels to decide what happens when a test fails.

2. Step 2: Identify the effect of ERROR severity

   When severity is set to ERROR, dbt stops the run immediately on failure.

3. Final Answer:

   It stops the dbt run if the test fails. -> Option A

4. Quick Check:

   ERROR severity = stops run [OK]

Hint: ERROR severity stops the run; WARN just warns [OK]

Common Mistakes:

• Confusing ERROR with WARN severity
• Thinking ERROR retries the test
• Assuming ERROR ignores failures

2. Which of the following is the correct way to set a test severity to WARN in a dbt YAML test configuration?

easy

A. severity: warn

B. severity = WARN

C. severity: WARN

D. severity: Warning

Solution

1. Step 1: Recall YAML syntax for dbt test severity

   dbt expects severity as a key-value pair with uppercase values like WARN or ERROR.

2. Step 2: Identify correct syntax

   The correct syntax uses a colon and uppercase WARN: severity: WARN.

3. Final Answer:

   severity: WARN -> Option C

4. Quick Check:

   YAML key-value with uppercase WARN = correct [OK]

Hint: Use colon and uppercase WARN for severity [OK]

Common Mistakes:

• Using lowercase 'warn' instead of uppercase
• Using equals sign instead of colon
• Spelling severity value incorrectly

3. Given this dbt test configuration snippet:

tests:
  - unique:
      column_name: id
      severity: WARN

What happens if the test fails during a dbt run?

medium

A. The failure is logged as a warning, but the run continues.

B. The test automatically retries until it passes.

C. The test is ignored and no message is shown.

D. The run stops immediately with an error.

Solution

1. Step 1: Analyze the severity level in the test config

   The severity is set to WARN, which means dbt should warn but not stop.

2. Step 2: Understand dbt behavior on WARN severity

   When a test fails with WARN severity, dbt logs a warning and continues the run.

3. Final Answer:

   The failure is logged as a warning, but the run continues. -> Option A

4. Quick Check:

   WARN severity = warn and continue [OK]

Hint: WARN severity warns but lets run continue [OK]

Common Mistakes:

• Thinking WARN stops the run
• Assuming WARN ignores failures
• Believing tests retry automatically

4. You wrote this test in your dbt model YAML:

tests:
  - not_null:
      column_name: user_id
      severity: ERROR

But your dbt run does not stop when the test fails. What is the likely problem?

medium

A. The test name 'not_null' is invalid.

B. The severity key is misplaced; it should be under 'config'.

C. The severity value should be lowercase 'error'.

D. The test is not actually failing.

Solution

1. Step 1: Check correct placement of severity in dbt tests

   Severity must be set inside the config block, not directly under the test.

2. Step 2: Identify why run does not stop

   Since severity is misplaced, dbt ignores it and uses default behavior, so run does not stop.

3. Final Answer:

   The severity key is misplaced; it should be under 'config'. -> Option B

4. Quick Check:

   Severity must be inside config = The severity key is misplaced; it should be under 'config'. [OK]

Hint: Put severity inside config block to take effect [OK]

Common Mistakes:

• Using lowercase severity value
• Misplacing severity outside config
• Assuming test name is wrong

5. You want to run a dbt test that warns on missing emails but errors on duplicate emails. Which YAML configuration correctly sets different severities for these tests on the email column?

hard

A. tests:
- not_null:
column_name: email
severity: WARN
- unique:
column_name: email
severity: ERROR

B. tests:
- not_null:
column_name: email
config:
severity: ERROR
- unique:
column_name: email
config:
severity: WARN

C. tests:
- not_null:
column_name: email
severity: ERROR
- unique:
column_name: email
severity: WARN

D. tests:
- not_null:
column_name: email
config:
severity: WARN
- unique:
column_name: email
config:
severity: ERROR

Solution

1. Step 1: Recall correct severity placement in dbt YAML

   Severity must be inside a config block under each test.

2. Step 2: Match severities to tests as required

   Set not_null severity to WARN and unique severity to ERROR inside their config blocks.

3. Final Answer:

   Tests with severity inside config: not_null WARN, unique ERROR. -> Option D

4. Quick Check:

   Severity inside config with correct WARN/ERROR = tests: - not_null: column_name: email config: severity: WARN - unique: column_name: email config: severity: ERROR [OK]

Hint: Use config block for different severities per test [OK]

Common Mistakes:

• Placing severity outside config block
• Swapping WARN and ERROR severities
• Using lowercase severity values