How to Use is_incremental Macro in dbt for Incremental Models
In dbt, use the
is_incremental() macro inside your model SQL to check if the model is running as an incremental build. This lets you write conditional logic to insert or update only new or changed data during incremental runs, improving performance.Syntax
The is_incremental() macro returns true if the current dbt run is incremental for that model, otherwise false. Use it inside a where clause or if statement to control incremental logic.
is_incremental(): Returns booleantrueorfalse.- Use inside
ifblocks orwhereclauses to filter data. - Works only in models configured with
materialized='incremental'.
jinja
{% if is_incremental() %}
-- incremental logic here
{% else %}
-- full-refresh logic here
{% endif %}Example
This example shows a dbt incremental model that inserts only new rows based on a timestamp column. It uses is_incremental() to apply a filter only during incremental runs.
sql
{{ config(materialized='incremental') }}
select * from source_table
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}Output
When run incrementally, only rows with updated_at newer than the max in the target table are inserted; on full refresh, all rows are selected.
Common Pitfalls
- Using
is_incremental()in models not set as incremental will always returnfalse. - Not applying a filter inside
if is_incremental()causes the entire dataset to reload, negating incremental benefits. - Forgetting to handle the full-refresh case can cause errors or duplicate data.
Always test both incremental and full-refresh runs.
jinja
Wrong usage:
{{ config(materialized='table') }}
select * from source_table
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}
-- This will never filter because materialization is not incremental.
Correct usage:
{{ config(materialized='incremental') }}
select * from source_table
{% if is_incremental() %}
where updated_at > (select max(updated_at) from {{ this }})
{% endif %}Quick Reference
| Macro | Description | Usage Context |
|---|---|---|
| is_incremental() | Returns true if model run is incremental | Inside incremental models only |
| if is_incremental() | Conditional logic for incremental runs | Wrap filters or inserts |
| else | Logic for full-refresh runs | Handle full data reloads |
Key Takeaways
Use
is_incremental() only in models with materialized='incremental'.Wrap incremental filters inside
if is_incremental() to load only new data.Always provide logic for both incremental and full-refresh runs to avoid errors.
Test your model with both incremental and full-refresh runs to ensure correctness.
Not using
is_incremental() properly can cause full data reloads or duplicates.