Why incremental models save time and cost in dbt - Visual Breakdown

Concept Flow - Why incremental models save time and cost

Start Full Model Build

↓

Process All Data Rows

↓

Store Full Result

↓

Next Run: Incremental Model

↓

Identify New/Changed Rows

↓

Process Only New/Changed Rows

↓

Append/Update Incremental Result

↓

Save Time and Cost

↓

End

Incremental models first build full data, then on next runs process only new or changed data, saving time and cost.

Execution Sample

dbt

select * from source_table
where updated_at > (select max(updated_at) from target_table)

This query selects only new or updated rows since last run to process incrementally.

Execution Table

Step	Action	Data Processed	Time Taken	Cost Impact
1	Full model build	All rows (100,000)	10 minutes	High cost
2	Store full result	100,000 rows saved	Instant	No extra cost
3	Next run starts	Trigger incremental	Instant	No cost yet
4	Identify new rows	5,000 new rows	1 minute	Low cost
5	Process new rows only	5,000 rows processed	1 minute	Low cost
6	Append new rows	Result updated	Instant	No extra cost
7	End incremental run	Total rows 105,000	2 minutes total	Low cost

💡 Incremental run stops after processing only new rows, saving time and cost compared to full build.

Variable Tracker

Variable	Start (Full Build)	After Incremental Run
Rows Processed	100,000	5,000
Time Taken (minutes)	10	2
Cost	High	Low

Key Moments - 3 Insights

Why does the incremental model process fewer rows on subsequent runs?

Because it only processes rows that are new or have changed since the last run, as shown in execution_table steps 4 and 5.

How does processing fewer rows save cost?

Less data processed means less compute time and resources used, lowering cost as seen in the reduced time and cost columns in the execution_table.

What happens if the incremental model runs a full build instead?

It would process all rows again, taking more time and cost, like step 1 in the execution_table.

Visual Quiz - 3 Questions

Look at the execution table, how many rows are processed during the incremental run?

A 105,000

B 100,000

C 5,000

D 0

💡 Hint

Check the 'Data Processed' column at step 5 in the execution_table.

At which step does the incremental model identify new or changed rows?

A Step 2

B Step 4

C Step 6

D Step 1

💡 Hint

Look at the 'Action' column in the execution_table for identifying new rows.

If the incremental model processed all rows every time, how would the 'Time Taken' change?

A It would increase to 10 minutes

B It would stay the same as 2 minutes

C It would decrease to 1 minute

D It would be zero

💡 Hint

Compare 'Time Taken' at step 1 and step 7 in the execution_table.

Concept Snapshot

Incremental models process only new or changed data after the first full build.
This reduces the amount of data processed each run.
Less data means less compute time and lower cost.
Use a filter on updated timestamps to select new rows.
Incremental saves resources and speeds up data workflows.

Full Transcript

Incremental models in dbt start by building the full dataset once. On later runs, they only process new or updated rows since the last run. This is done by filtering data using a timestamp or similar marker. Processing fewer rows saves time and reduces compute costs. The execution table shows a full build processing 100,000 rows taking 10 minutes, while the incremental run processes only 5,000 rows in 2 minutes. This approach is efficient and cost-effective for large datasets.