Why incremental models save time and cost in dbt

Introduction

Incremental models only update new or changed data instead of rebuilding everything. This saves time and computing resources.

When you have a large dataset that grows over time.

When you want to update your data model daily without reprocessing all data.

When you want to reduce cloud computing costs by processing less data.

When your data source adds new records regularly but old data rarely changes.

When you want faster data refreshes for reports or dashboards.

Syntax

dbt

{{ config(
  materialized='incremental',
  unique_key='id'
) }}

select * from source_table
{% if is_incremental() %}
  where updated_at > (select max(updated_at) from {{ this }})
{% endif %}

The materialized='incremental' tells dbt to build the model incrementally.

The is_incremental() function helps filter only new or changed rows during incremental runs.

Examples

Incrementally load new orders based on the latest order date.

dbt

{{ config(materialized='incremental', unique_key='order_id') }}

select * from orders
{% if is_incremental() %}
  where order_date > (select max(order_date) from {{ this }})
{% endif %}

Incrementally update user records changed since last run.

dbt

{{ config(materialized='incremental', unique_key='user_id') }}

select * from users
{% if is_incremental() %}
  where last_modified > (select max(last_modified) from {{ this }})
{% endif %}

Sample Program

This example shows an incremental model that loads only new rows with updated dates after the last run.

dbt

{{ config(materialized='incremental', unique_key='id') }}

with source_data as (
  select * from (values
    (1, '2024-01-01'),
    (2, '2024-01-02'),
    (3, '2024-01-03')
  ) as t(id, updated_at)
)

select * from source_data
{% if is_incremental() %}
  where updated_at > (select max(updated_at) from {{ this }})
{% endif %}

Important Notes

Incremental models reduce processing time by avoiding full rebuilds.

Make sure your unique key and filter logic correctly identify new or changed data.

Test incremental logic carefully to avoid missing or duplicating data.

Summary

Incremental models update only new or changed data.

This saves time and lowers cloud computing costs.

Use is_incremental() to filter data during incremental runs.