How dbt works (SQL + Jinja + YAML)

Introduction

dbt helps you organize and run your data transformations easily. It uses SQL for queries, Jinja to add logic, and YAML to manage settings.

You want to build clean, reusable data models from raw data.

You need to automate running SQL queries with dynamic parts.

You want to document and test your data models clearly.

You want to manage data sources and model configurations in one place.

You want to track dependencies between data transformations.

Syntax

dbt

model.sql
-- SQL code with Jinja templating

models.yml
# YAML file for configuration and documentation

SQL files contain your data transformation queries.

Jinja lets you add variables, loops, and conditions inside SQL.

Examples

This SQL uses Jinja to get data from a source table named 'users' in the 'raw' schema.

dbt

-- model.sql
SELECT * FROM {{ source('raw', 'users') }} WHERE active = true

Here, Jinja sets a variable 'cutoff' and uses it inside the SQL query.

dbt

{% set cutoff = '2023-01-01' %}
SELECT * FROM sales WHERE sale_date >= '{{ cutoff }}'

This YAML documents the 'users' model and its columns.

dbt

models.yml
version: 2
models:
  - name: users
    description: 'User details table'
    columns:
      - name: id
        description: 'User ID'
      - name: active
        description: 'If user is active'

Sample Program

This example shows a dbt model SQL file using Jinja to filter active users. The YAML file documents the model and its columns.

dbt

-- models/users.sql
{% set active_only = true %}
SELECT id, name, email
FROM {{ source('raw', 'users') }}
{% if active_only %} WHERE active = true {% endif %}

-- models.yml
version: 2
models:
  - name: users
    description: 'Filtered active users'
    columns:
      - name: id
        description: 'User ID'
      - name: name
        description: 'User name'
      - name: email
        description: 'User email address'

Important Notes

dbt runs Jinja first to create the final SQL query before running it.

YAML files help with documentation and testing but do not run SQL.

Using Jinja makes your SQL flexible and reusable.

Summary

dbt combines SQL, Jinja, and YAML to build, run, and document data models.

SQL writes the queries, Jinja adds logic, YAML manages configs and docs.

This makes data transformation easier, clearer, and more maintainable.

Practice

1. What is the main role of Jinja in dbt projects?

easy

A. To add logic and dynamic behavior to SQL queries

B. To write raw SQL queries without any modification

C. To manage configuration and documentation files

D. To execute the SQL queries on the database

Solution

1. Step 1: Understand Jinja's purpose in dbt

   Jinja is a templating language that allows adding logic like loops and conditions inside SQL files.

2. Step 2: Differentiate roles of SQL, Jinja, and YAML

   SQL writes queries, YAML manages configs/docs, and Jinja adds dynamic logic to SQL.

3. Final Answer:

   To add logic and dynamic behavior to SQL queries -> Option A

4. Quick Check:

   Jinja = logic in SQL [OK]

Hint: Jinja = logic inside SQL, YAML = configs/docs [OK]

Common Mistakes:

• Confusing Jinja with YAML for configs
• Thinking Jinja executes SQL queries
• Assuming Jinja writes raw SQL without changes

2. Which of the following is the correct way to use a Jinja variable inside a dbt SQL model?

easy

A. SELECT * FROM var('table_name')

B. SELECT * FROM {{ var('table_name') }}

C. SELECT * FROM {% var('table_name') %}

D. SELECT * FROM [[ var('table_name') ]]

Solution

1. Step 1: Recall Jinja syntax for variables

   Jinja variables are inserted using double curly braces {{ }} around expressions.

2. Step 2: Identify correct syntax for var function

   The correct syntax is {{ var('variable_name') }} to access a variable in dbt.

3. Final Answer:

   SELECT * FROM {{ var('table_name') }} -> Option B

4. Quick Check:

   Jinja variables use {{ }} [OK]

Hint: Use {{ var('name') }} to insert variables in SQL [OK]

Common Mistakes:

• Using single curly braces or wrong brackets
• Confusing Jinja tags {% %} with variable insertion {{ }}
• Using square brackets instead of curly braces

3. Given this dbt model SQL code, what will be the output SQL after rendering?

SELECT
  user_id,
  {% if var('include_email', false) %}
    email,
  {% endif %}
  created_at
FROM users

Assuming the variable include_email is set to true in dbt_project.yml.

medium

A. SELECT user_id, true, created_at FROM users

B. SELECT user_id, created_at FROM users

C. Syntax error due to misplaced Jinja

D. SELECT user_id, email, created_at FROM users

Solution

1. Step 1: Check the value of the variable include_email

   The variable include_email is true, so the if condition passes and the email column is included.

2. Step 2: Render the SQL with the if block included

   The SQL will have user_id, email, and created_at columns selected from users.

3. Final Answer:

   SELECT user_id, email, created_at FROM users -> Option D

4. Quick Check:

   include_email true means email included [OK]

Hint: If var true, include block inside {% if %} [OK]

Common Mistakes:

• Ignoring the variable value and excluding email
• Thinking Jinja syntax causes SQL errors
• Confusing variable default values

4. You wrote this YAML config in your dbt project:

models:
  my_project:
    +materialized: table
      users:
        +tags: ['important']

Why does dbt raise an error when running?

medium

A. Because the indentation for 'users' is incorrect under 'my_project'

B. Because '+materialized' cannot be set in YAML

C. Because tags must be a string, not a list

D. Because 'models' key is missing

Solution

1. Step 1: Check YAML indentation rules for dbt configs

   In dbt, model configs under a project must be indented properly; 'users' should be at the same level as '+materialized'.

2. Step 2: Identify the indentation error

   'users' is indented too far, making it a child of '+materialized' which is invalid.

3. Final Answer:

   Because the indentation for 'users' is incorrect under 'my_project' -> Option A

4. Quick Check:

   YAML indentation matters for nested configs [OK]

Hint: Check YAML indentation carefully for nested configs [OK]

Common Mistakes:

• Ignoring YAML indentation importance
• Thinking '+materialized' is invalid syntax
• Assuming tags cannot be lists

5. You want to create a dbt model that selects only active users from a table, but the 'active' flag is stored in a YAML config. Which approach correctly combines SQL, Jinja, and YAML to achieve this?

hard

A. Use Jinja to read YAML directly inside SQL without defining variables

B. Write WHERE active = true directly in SQL without YAML or Jinja

C. Define 'active_flag: true' in YAML, then use WHERE active = {{ var('active_flag') }} in SQL with Jinja

D. Set 'active_flag' in YAML but forget to use Jinja in SQL, so filter is missing

Solution

1. Step 1: Store the filter value in YAML as a variable

   Define 'active_flag: true' in YAML under vars or config to make it accessible.

2. Step 2: Use Jinja to insert the variable in SQL WHERE clause

   Use WHERE active = {{ var('active_flag') }} so the SQL filters active users dynamically.

3. Final Answer:

   Define 'active_flag: true' in YAML, then use WHERE active = {{ var('active_flag') }} in SQL with Jinja -> Option C

4. Quick Check:

   YAML vars + Jinja in SQL = dynamic filters [OK]

Hint: Use YAML vars + Jinja {{ var() }} in SQL WHERE [OK]

Common Mistakes:

• Hardcoding filter in SQL ignoring YAML
• Not using Jinja to insert YAML vars
• Trying to read YAML directly in SQL without var()