How dbt works (SQL + Jinja + YAML) - Performance & Efficiency

Time Complexity: How dbt works (SQL + Jinja + YAML)

O(n)

Understanding Time Complexity

We want to understand how the time dbt takes to run grows as the size of data or number of models increases.

Specifically, how does dbt's combination of SQL, Jinja, and YAML affect execution time?

Scenario Under Consideration

Analyze the time complexity of this dbt model snippet.

-- models/example_model.sql
{{ config(materialized='table') }}

select
  user_id,
  count(*) as total_orders
from {{ ref('orders') }}
where order_date >= '{{ var("start_date") }}'
group by user_id

This code runs a SQL query with Jinja templating and YAML config to build a table from a referenced model.

Identify Repeating Operations

Look at what repeats as input grows.

• Primary operation: Scanning and grouping rows in the referenced table.
• How many times: Once per run, but the scan touches every row matching the date filter.

How Execution Grows With Input

As the number of rows in the orders table grows, the query scans more data.

Input Size (n rows)	Approx. Operations
10	10 rows scanned and grouped
100	100 rows scanned and grouped
1000	1000 rows scanned and grouped

Pattern observation: The work grows roughly in direct proportion to the number of rows scanned.

Final Time Complexity

Time Complexity: O(n)

This means the time grows linearly with the number of rows processed in the SQL query.

Common Mistake

[X] Wrong: "dbt runs all models instantly regardless of data size because it just runs SQL."

[OK] Correct: The SQL query inside dbt still processes data, so bigger tables mean more work and longer run times.

Interview Connect

Understanding how dbt runs SQL with templating helps you explain data pipeline performance clearly and confidently.

Self-Check

"What if the model used a more complex join instead of a simple filter? How would the time complexity change?"

Practice

1. What is the main role of Jinja in dbt projects?

easy

A. To add logic and dynamic behavior to SQL queries

B. To write raw SQL queries without any modification

C. To manage configuration and documentation files

D. To execute the SQL queries on the database

Solution

1. Step 1: Understand Jinja's purpose in dbt

   Jinja is a templating language that allows adding logic like loops and conditions inside SQL files.

2. Step 2: Differentiate roles of SQL, Jinja, and YAML

   SQL writes queries, YAML manages configs/docs, and Jinja adds dynamic logic to SQL.

3. Final Answer:

   To add logic and dynamic behavior to SQL queries -> Option A

4. Quick Check:

   Jinja = logic in SQL [OK]

Hint: Jinja = logic inside SQL, YAML = configs/docs [OK]

Common Mistakes:

• Confusing Jinja with YAML for configs
• Thinking Jinja executes SQL queries
• Assuming Jinja writes raw SQL without changes

2. Which of the following is the correct way to use a Jinja variable inside a dbt SQL model?

easy

A. SELECT * FROM var('table_name')

B. SELECT * FROM {{ var('table_name') }}

C. SELECT * FROM {% var('table_name') %}

D. SELECT * FROM [[ var('table_name') ]]

Solution

1. Step 1: Recall Jinja syntax for variables

   Jinja variables are inserted using double curly braces {{ }} around expressions.

2. Step 2: Identify correct syntax for var function

   The correct syntax is {{ var('variable_name') }} to access a variable in dbt.

3. Final Answer:

   SELECT * FROM {{ var('table_name') }} -> Option B

4. Quick Check:

   Jinja variables use {{ }} [OK]

Hint: Use {{ var('name') }} to insert variables in SQL [OK]

Common Mistakes:

• Using single curly braces or wrong brackets
• Confusing Jinja tags {% %} with variable insertion {{ }}
• Using square brackets instead of curly braces

3. Given this dbt model SQL code, what will be the output SQL after rendering?

SELECT
  user_id,
  {% if var('include_email', false) %}
    email,
  {% endif %}
  created_at
FROM users

Assuming the variable include_email is set to true in dbt_project.yml.

medium

A. SELECT user_id, true, created_at FROM users

B. SELECT user_id, created_at FROM users

C. Syntax error due to misplaced Jinja

D. SELECT user_id, email, created_at FROM users

Solution

1. Step 1: Check the value of the variable include_email

   The variable include_email is true, so the if condition passes and the email column is included.

2. Step 2: Render the SQL with the if block included

   The SQL will have user_id, email, and created_at columns selected from users.

3. Final Answer:

   SELECT user_id, email, created_at FROM users -> Option D

4. Quick Check:

   include_email true means email included [OK]

Hint: If var true, include block inside {% if %} [OK]

Common Mistakes:

• Ignoring the variable value and excluding email
• Thinking Jinja syntax causes SQL errors
• Confusing variable default values

4. You wrote this YAML config in your dbt project:

models:
  my_project:
    +materialized: table
      users:
        +tags: ['important']

Why does dbt raise an error when running?

medium

A. Because the indentation for 'users' is incorrect under 'my_project'

B. Because '+materialized' cannot be set in YAML

C. Because tags must be a string, not a list

D. Because 'models' key is missing

Solution

1. Step 1: Check YAML indentation rules for dbt configs

   In dbt, model configs under a project must be indented properly; 'users' should be at the same level as '+materialized'.

2. Step 2: Identify the indentation error

   'users' is indented too far, making it a child of '+materialized' which is invalid.

3. Final Answer:

   Because the indentation for 'users' is incorrect under 'my_project' -> Option A

4. Quick Check:

   YAML indentation matters for nested configs [OK]

Hint: Check YAML indentation carefully for nested configs [OK]

Common Mistakes:

• Ignoring YAML indentation importance
• Thinking '+materialized' is invalid syntax
• Assuming tags cannot be lists

5. You want to create a dbt model that selects only active users from a table, but the 'active' flag is stored in a YAML config. Which approach correctly combines SQL, Jinja, and YAML to achieve this?

hard

A. Use Jinja to read YAML directly inside SQL without defining variables

B. Write WHERE active = true directly in SQL without YAML or Jinja

C. Define 'active_flag: true' in YAML, then use WHERE active = {{ var('active_flag') }} in SQL with Jinja

D. Set 'active_flag' in YAML but forget to use Jinja in SQL, so filter is missing

Solution

1. Step 1: Store the filter value in YAML as a variable

   Define 'active_flag: true' in YAML under vars or config to make it accessible.

2. Step 2: Use Jinja to insert the variable in SQL WHERE clause

   Use WHERE active = {{ var('active_flag') }} so the SQL filters active users dynamically.

3. Final Answer:

   Define 'active_flag: true' in YAML, then use WHERE active = {{ var('active_flag') }} in SQL with Jinja -> Option C

4. Quick Check:

   YAML vars + Jinja in SQL = dynamic filters [OK]

Hint: Use YAML vars + Jinja {{ var() }} in SQL WHERE [OK]

Common Mistakes:

• Hardcoding filter in SQL ignoring YAML
• Not using Jinja to insert YAML vars
• Trying to read YAML directly in SQL without var()