is_incremental() macro in dbt - Step-by-Step Execution

Concept Flow - is_incremental() macro

Start dbt model run

↓

Check if run is incremental?

↓

Run incremental

↓

Append new data

↓

End run

The is_incremental() macro checks if the current dbt model run is incremental or full refresh, guiding the model to append new data or replace all data.

Execution Sample

dbt

select * from source_table
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}

This code selects all data from source_table, but if running incrementally, it only selects new or updated rows.

Execution Table

Step	is_incremental() Result	Condition Checked	Action Taken	Output Description
1	True	Is this an incremental run?	Apply WHERE filter to select new rows	Only rows with updated_at > max(updated_at) in target are selected
2	False	Is this an incremental run?	No filter applied, select all rows	All rows from source_table are selected
3	True	Run incremental append	Append new rows to existing table	Table grows with new data
4	False	Run full refresh	Replace entire table with new data	Table is fully replaced
5	-	Run ends	Process complete	Model run finished

💡 Execution stops after data is either appended (incremental) or fully replaced (full refresh).

Variable Tracker

Variable	Start	After Step 1	After Step 3	Final
is_incremental()	undefined	True or False depending on run type	True or False	True or False
selected_rows	empty	Filtered rows if incremental, else all rows	Rows ready to append or replace	Final rows loaded into table

Key Moments - 2 Insights

Why does the model select fewer rows when is_incremental() returns True?

Because when is_incremental() is True, the model adds a WHERE clause to select only new or updated rows, as shown in execution_table row 1.

What happens if is_incremental() returns False during a run?

The model runs a full refresh, selecting all rows and replacing the existing table, as shown in execution_table rows 2 and 4.

Visual Quiz - 3 Questions

Look at the execution table, what action is taken when is_incremental() returns True?

A Replace entire table with new data

B Skip data loading

C Append new rows to existing table

D Delete existing data without loading

💡 Hint

See execution_table row 3 for the action when is_incremental() is True.

At which step does the model decide to run a full refresh?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Check execution_table row 2 where is_incremental() is False.

If the WHERE clause inside the if block is removed, how does the output change when is_incremental() is True?

A All rows are selected regardless of incremental run

B No rows are selected

C Only new rows are selected

D Model run fails

💡 Hint

Refer to execution_sample code where the WHERE clause filters rows only if is_incremental() is True.

Concept Snapshot

is_incremental() macro in dbt:
- Returns True if the model run is incremental
- Use inside {% if is_incremental() %} to filter new data
- Incremental runs append data, full refresh replaces
- Helps optimize data loading by avoiding full reloads
- Commonly used with WHERE clauses to select new/updated rows

Full Transcript

The is_incremental() macro in dbt helps the model know if it is running incrementally or as a full refresh. When running incrementally, the macro returns True, so the model can add a filter to select only new or updated rows. This way, the model appends only new data instead of reloading everything. If the macro returns False, the model runs a full refresh and replaces all data. This process optimizes data loading and saves time. The execution table shows steps where the macro result guides the model's behavior, either filtering rows or selecting all. Variables like is_incremental() and selected_rows change accordingly during the run. Understanding this macro helps beginners write efficient dbt models.