Staging, intermediate, and marts pattern in dbt

Introduction

This pattern helps organize data work in steps. It makes data easier to clean, transform, and use for reports.

When you get raw data from many sources and want to clean it first.

When you need to combine and transform data before final use.

When you want to build clear, reusable data layers for reports or dashboards.

When you want to keep your data work organized and easy to update.

When you want to separate simple cleaning from complex business logic.

Syntax

dbt

models/
  staging/
    source_table.sql
  intermediate/
    transformed_table.sql
  marts/
    final_report_table.sql

Staging: Raw data cleaning and simple fixes.

Intermediate: Data transformations and combining tables.

Examples

Staging model just selects raw data for cleaning later.

dbt

-- models/staging/customers.sql
select * from raw.customers

Intermediate model joins staging tables to combine data.

dbt

-- models/intermediate/customer_orders.sql
select c.id, c.name, o.order_date
from {{ ref('customers') }} c
join {{ ref('orders') }} o on c.id = o.customer_id

Marts model aggregates data for final reports.

dbt

-- models/marts/sales_report.sql
select customer_id, count(*) as total_orders
from {{ ref('customer_orders') }}
group by customer_id

Sample Program

This example shows three layers: staging cleans raw tables, intermediate joins them, and marts summarize data for reports.

dbt

-- models/staging/customers.sql
select id, name, email from raw.customers

-- models/staging/orders.sql
select id, customer_id, order_date from raw.orders

-- models/intermediate/customer_orders.sql
select c.id as customer_id, c.name, o.order_date
from {{ ref('customers') }} c
join {{ ref('orders') }} o on c.id = o.customer_id

-- models/marts/customer_order_counts.sql
select customer_id, count(*) as total_orders
from {{ ref('customer_orders') }}
group by customer_id

Important Notes

Use {{ ref('model_name') }} to link models and keep dependencies clear.

Keep staging models simple to make debugging easier.

Intermediate models handle business logic and complex joins.

Summary

The pattern splits data work into clear steps: staging, intermediate, and marts.

This helps keep data clean, organized, and easy to maintain.

Use dbt refs to connect models and build a reliable data pipeline.

Practice

1. What is the main purpose of the staging layer in the dbt pattern Staging, intermediate, and marts?

easy

A. To create final reports and dashboards

B. To aggregate data for business metrics

C. To clean and prepare raw data for further transformations

D. To store backup copies of raw data

Solution

1. Step 1: Understand the role of staging

   The staging layer is where raw data is cleaned and standardized to make it ready for analysis.

2. Step 2: Differentiate from other layers

   Intermediate and marts layers perform transformations and aggregations, while staging focuses on cleaning.

3. Final Answer:

   To clean and prepare raw data for further transformations -> Option C

4. Quick Check:

   Staging = cleaning raw data [OK]

Hint: Staging = clean raw data first [OK]

Common Mistakes:

• Confusing staging with marts layer
• Thinking staging creates final reports
• Assuming staging stores backups

2. Which of the following is the correct way to reference a model in dbt within the intermediate layer?

easy

A. SELECT * FROM stg_customers

B. SELECT * FROM {{ ref('stg_customers') }}

C. SELECT * FROM ref('stg_customers')

D. SELECT * FROM {{ source('stg_customers') }}

Solution

1. Step 1: Recall dbt model referencing syntax

   dbt uses the Jinja function ref() inside double curly braces to reference models.

2. Step 2: Identify correct syntax

   The correct syntax is {{ ref('model_name') }}, so {{ ref('stg_customers') }} is correct.

3. Final Answer:

   SELECT * FROM {{ ref('stg_customers') }} -> Option B

4. Quick Check:

   Use {{ ref('model') }} to reference models [OK]

Hint: Use {{ ref('model_name') }} to reference models [OK]

Common Mistakes:

• Omitting curly braces around ref()
• Using source() instead of ref() for models
• Calling ref() without curly braces

3. Given the following dbt model code in the intermediate layer:

SELECT
  customer_id,
  COUNT(order_id) AS total_orders
FROM {{ ref('stg_orders') }}
GROUP BY customer_id

What will be the output columns of this model?

medium

A. customer_id, total_orders

B. order_id, total_orders

C. customer_id, order_id

D. customer_id only

Solution

1. Step 1: Analyze SELECT statement

   The query selects customer_id and counts order_id as total_orders.

2. Step 2: Understand GROUP BY effect

   Grouping by customer_id means output will have one row per customer with total orders counted.

3. Final Answer:

   customer_id, total_orders -> Option A

4. Quick Check:

   Output columns = customer_id and total_orders [OK]

Hint: GROUP BY columns + aggregated columns appear in output [OK]

Common Mistakes:

• Expecting order_id as output column
• Missing aggregated column in output
• Confusing GROUP BY with WHERE clause

4. You wrote this dbt model in the marts layer:

SELECT
  customer_id,
  SUM(total_orders) AS total_orders
FROM {{ ref('int_customer_orders') }}
GROUP BY customer_id

But the model fails with an error about total_orders. What is the likely cause?

medium

A. Missing GROUP BY clause

B. SUM() cannot be used in marts layer

C. Incorrect use of {{ ref() }} syntax

D. The referenced model int_customer_orders does not have a column named total_orders

Solution

1. Step 1: Check referenced model columns

   The error suggests total_orders column is missing in int_customer_orders.

2. Step 2: Validate aggregation usage

   SUM() is valid in marts layer and GROUP BY is present, so those are not issues.

3. Final Answer:

   The referenced model int_customer_orders does not have a column named total_orders -> Option D

4. Quick Check:

   Column missing in referenced model causes error [OK]

Hint: Check referenced model columns before aggregating [OK]

Common Mistakes:

• Assuming SUM() is invalid in marts
• Forgetting GROUP BY with aggregation
• Misusing ref() syntax

5. You want to build a sales report in the marts layer that shows total sales per region. Your staging layer has raw sales data, and the intermediate layer calculates sales per store. Which is the best approach to build this report using the staging, intermediate, and marts pattern?

hard

A. Reference the intermediate sales per store model in the marts layer and aggregate sales by region

B. Directly query the raw sales data in the marts layer and group by region

C. Aggregate sales by region in the staging layer

D. Create the sales report in the intermediate layer without using marts

Solution

1. Step 1: Follow pattern responsibilities

   Staging cleans raw data, intermediate transforms data per store, marts create final aggregates like sales per region.

2. Step 2: Use intermediate model in marts

   Use the intermediate sales per store model as input to marts, then aggregate by region for the report.

3. Final Answer:

   Reference the intermediate sales per store model in the marts layer and aggregate sales by region -> Option A

4. Quick Check:

   Marts aggregate intermediate data for final reports [OK]

Hint: Build reports in marts using intermediate models [OK]

Common Mistakes:

• Skipping intermediate layer aggregation
• Aggregating raw data directly in marts
• Doing final aggregation in staging