Snapshot tables for historical tracking in dbt

Introduction

Snapshot tables help you keep a history of data changes over time. They let you see what data looked like at different points.

Tracking price changes of products over months.

Keeping a record of customer address updates.

Monitoring employee role changes in a company.

Saving daily status of orders for analysis.

Auditing changes in financial transactions.

Syntax

dbt

snapshots:
  - name: your_snapshot_name
    target_schema: your_schema
    strategy: timestamp
    updated_at: updated_at_column
    unique_key: id_column
    check_cols: ['column1', 'column2']
    source: your_source_table

strategy: Choose timestamp to track changes based on a timestamp column or check to track changes by comparing columns.

unique_key: This is the column that identifies each record uniquely.

Examples

This snapshot tracks product price changes using the last_updated timestamp.

dbt

snapshots:
  - name: product_price_snapshot
    target_schema: analytics
    strategy: timestamp
    updated_at: last_updated
    unique_key: product_id
    check_cols: ['price']
    source: raw.products

This snapshot tracks changes in customer addresses by checking if any address fields changed.

dbt

snapshots:
  - name: customer_address_snapshot
    target_schema: analytics
    strategy: check
    unique_key: customer_id
    check_cols: ['address', 'city', 'zip']
    source: raw.customers

Sample Program

This snapshot tracks changes in employee roles by comparing the role column. When you run dbt snapshot, it creates a table that keeps all historical role changes.

dbt

version: 2

snapshots:
  - name: employee_role_snapshot
    target_schema: analytics
    strategy: check
    unique_key: employee_id
    check_cols: ['role']
    source: raw.employees

-- Run this snapshot with dbt snapshot command to create and update the snapshot table.

Important Notes

Snapshots require a unique key to identify records.

Choose timestamp strategy if you have a reliable updated timestamp column.

Choose check strategy if you want to track changes by comparing specific columns.

Summary

Snapshot tables keep historical versions of data for tracking changes.

Use timestamp or check strategies depending on your data.

Run dbt snapshot to create and update snapshot tables.

Practice

1. What is the main purpose of using snapshot tables in dbt?

easy

A. To save historical versions of data automatically

B. To speed up query performance by caching results

C. To create temporary tables for intermediate calculations

D. To delete old data from the database

Solution

1. Step 1: Understand snapshot tables

   Snapshot tables store copies of data at different points in time to track changes.

2. Step 2: Identify the main use

   They are used to save historical versions automatically, not for caching or deleting data.

3. Final Answer:

   To save historical versions of data automatically -> Option A

4. Quick Check:

   Snapshot tables = historical data tracking [OK]

Hint: Snapshots track data changes over time automatically [OK]

Common Mistakes:

• Confusing snapshots with caching mechanisms
• Thinking snapshots delete old data
• Assuming snapshots create temporary tables

2. Which of the following is the correct way to define a snapshot strategy in dbt?

easy

A. strategy: timestamp

B. method: snapshot

C. type: incremental

D. mode: historical

Solution

1. Step 1: Recall snapshot config syntax

   In dbt snapshots, the strategy is set using strategy: timestamp or strategy: check.

2. Step 2: Identify correct keyword

   Only strategy: timestamp is valid; others like method, type, or mode are incorrect.

3. Final Answer:

   strategy: timestamp -> Option A

4. Quick Check:

   Snapshot strategy keyword = strategy [OK]

Hint: Use 'strategy' keyword to set snapshot method [OK]

Common Mistakes:

• Using wrong keywords like 'method' or 'type'
• Confusing snapshot strategy with model materialization
• Misspelling 'strategy'

3. Given this snapshot config snippet:

snapshots:
  - name: customer_snapshot
    strategy: check
    check_cols: ['email', 'status']

What happens when the email or status changes in the source data?

medium

A. The snapshot process ignores changes in these columns

B. The existing record is overwritten without history

C. A new record version is created in the snapshot table

D. An error occurs because check_cols is invalid

Solution

1. Step 1: Understand 'check' strategy with check_cols

   The 'check' strategy tracks changes in specified columns and creates new versions when they change.

2. Step 2: Apply to given columns

   If 'email' or 'status' changes, dbt adds a new record version to keep history.

3. Final Answer:

   A new record version is created in the snapshot table -> Option C

4. Quick Check:

   Check strategy tracks column changes = new version [OK]

Hint: Check strategy creates new versions on column changes [OK]

Common Mistakes:

• Thinking existing records get overwritten
• Believing changes are ignored
• Assuming check_cols is not valid syntax

4. You wrote this snapshot config:

snapshots:
  - name: order_snapshot
    strategy: timestamp
    updated_at: updated_at_column

But dbt throws an error. What is the likely cause?

medium

A. The 'updated_at' key should be inside 'config' block

B. The 'strategy' value 'timestamp' is invalid

C. The 'updated_at' key is missing quotes

D. The 'updated_at' key should be 'updated_at_column'

Solution

1. Step 1: Check snapshot config keys for timestamp strategy

   For 'timestamp' strategy, the correct key is updated_at_column, not updated_at.

2. Step 2: Identify the error cause

   Using updated_at causes dbt to error because it expects updated_at_column.

3. Final Answer:

   The 'updated_at' key should be 'updated_at_column' -> Option D

4. Quick Check:

   Timestamp strategy key = updated_at_column [OK]

Hint: Use 'updated_at_column' key for timestamp strategy [OK]

Common Mistakes:

• Using 'updated_at' instead of 'updated_at_column'
• Misplacing keys outside config block
• Assuming strategy value is wrong

5. You want to track changes in a product price over time using dbt snapshots. The source table has columns product_id, price, and last_updated. Which snapshot config is best to capture price changes accurately?

hard

A. strategy: timestamp with updated_at_column: last_updated

B. strategy: check with check_cols: ['price']

C. strategy: timestamp with updated_at_column: price

D. strategy: check with check_cols: ['last_updated']

Solution

1. Step 1: Understand the difference between timestamp and check strategies

   Timestamp strategy uses a datetime column to detect changes; check strategy compares specified columns directly.

2. Step 2: Analyze columns for tracking price changes

   Price changes may not always update 'last_updated' reliably, so checking 'price' column directly is safer.

3. Step 3: Choose the best config

   Using strategy: check with check_cols: ['price'] ensures any price change triggers a new snapshot version.

4. Final Answer:

   strategy: check with check_cols: ['price'] -> Option B

5. Quick Check:

   Check strategy on price column = accurate price tracking [OK]

Hint: Use check strategy on price column for accurate tracking [OK]

Common Mistakes:

• Using timestamp on a non-datetime column
• Relying on last_updated which may be unreliable
• Checking wrong columns for changes