Staging, intermediate, and marts pattern in dbt - Mini Project: Build & Apply

Staging, Intermediate, and Marts Pattern in dbt

📖 Scenario: You work as a data analyst at a retail company. You receive raw sales data from multiple sources. To make this data useful for business decisions, you need to organize it in layers: staging, intermediate, and marts.Staging cleans and standardizes raw data. Intermediate transforms and combines staging data. Marts prepare final tables for reporting.

🎯 Goal: Build a simple dbt project with three models: a staging model to clean raw sales data, an intermediate model to calculate monthly sales totals, and a marts model to prepare the final sales report.

📋 What You'll Learn

Create a staging model called stg_sales that selects and cleans raw sales data

Create an intermediate model called int_monthly_sales that aggregates sales by month

Create a marts model called mart_sales_report that selects final columns for reporting

Use dbt model files with correct SQL SELECT statements

Use appropriate naming conventions for each layer

💡 Why This Matters

🌍 Real World

Data teams use this pattern to build reliable, maintainable data pipelines that transform raw data into business-ready reports.

💼 Career

Understanding this pattern is essential for data analysts, engineers, and scientists working with dbt or modern data warehouses.

1

Create the staging model stg_sales.sql

Create a dbt model file named stg_sales.sql. Write a SQL SELECT statement that selects order_id, order_date, customer_id, and amount from the raw table raw_sales. Rename amount to sale_amount. This model cleans and standardizes the raw data.

dbt

-- Your code here to create stg_sales.sql
-- Select order_id, order_date, customer_id, and amount as sale_amount from raw_sales

Hint

Use SELECT to pick columns and AS to rename amount to sale_amount.

2

Create the intermediate model int_monthly_sales.sql

Create a dbt model file named int_monthly_sales.sql. Write a SQL SELECT statement that selects the year and month from order_date as order_year and order_month, and sums sale_amount as total_sales. Use the staging model stg_sales as the source table. Group by year and month.

dbt

select
  -- Extract year and month from order_date
  -- Sum sale_amount as total_sales
from {{ ref('stg_sales') }}
-- Group by year and month
-- Your code here

Hint

Use extract to get year and month, sum to aggregate, and {{ ref('stg_sales') }} to reference the staging model.

3

Create the marts model mart_sales_report.sql

Create a dbt model file named mart_sales_report.sql. Write a SQL SELECT statement that selects order_year, order_month, and total_sales from the intermediate model int_monthly_sales. This model prepares the final sales report.

dbt

select
  order_year,
  order_month,
  total_sales
from {{ ref('int_monthly_sales') }}
-- Your code here

Hint

Simply select the columns from the intermediate model using {{ ref('int_monthly_sales') }}.

4

Run and display the final sales report

Write a SQL SELECT statement to query all columns from the marts model mart_sales_report. This will display the final sales report with year, month, and total sales.

dbt

select * from {{ ref('mart_sales_report') }}
-- Your code here

Hint

Use select * from the marts model to see the final report.

Practice

1. What is the main purpose of the staging layer in the dbt pattern Staging, intermediate, and marts?

easy

A. To create final reports and dashboards

B. To aggregate data for business metrics

C. To clean and prepare raw data for further transformations

D. To store backup copies of raw data

Solution

1. Step 1: Understand the role of staging

   The staging layer is where raw data is cleaned and standardized to make it ready for analysis.

2. Step 2: Differentiate from other layers

   Intermediate and marts layers perform transformations and aggregations, while staging focuses on cleaning.

3. Final Answer:

   To clean and prepare raw data for further transformations -> Option C

4. Quick Check:

   Staging = cleaning raw data [OK]

Hint: Staging = clean raw data first [OK]

Common Mistakes:

• Confusing staging with marts layer
• Thinking staging creates final reports
• Assuming staging stores backups

2. Which of the following is the correct way to reference a model in dbt within the intermediate layer?

easy

A. SELECT * FROM stg_customers

B. SELECT * FROM {{ ref('stg_customers') }}

C. SELECT * FROM ref('stg_customers')

D. SELECT * FROM {{ source('stg_customers') }}

Solution

1. Step 1: Recall dbt model referencing syntax

   dbt uses the Jinja function ref() inside double curly braces to reference models.

2. Step 2: Identify correct syntax

   The correct syntax is {{ ref('model_name') }}, so {{ ref('stg_customers') }} is correct.

3. Final Answer:

   SELECT * FROM {{ ref('stg_customers') }} -> Option B

4. Quick Check:

   Use {{ ref('model') }} to reference models [OK]

Hint: Use {{ ref('model_name') }} to reference models [OK]

Common Mistakes:

• Omitting curly braces around ref()
• Using source() instead of ref() for models
• Calling ref() without curly braces

3. Given the following dbt model code in the intermediate layer:

SELECT
  customer_id,
  COUNT(order_id) AS total_orders
FROM {{ ref('stg_orders') }}
GROUP BY customer_id

What will be the output columns of this model?

medium

A. customer_id, total_orders

B. order_id, total_orders

C. customer_id, order_id

D. customer_id only

Solution

1. Step 1: Analyze SELECT statement

   The query selects customer_id and counts order_id as total_orders.

2. Step 2: Understand GROUP BY effect

   Grouping by customer_id means output will have one row per customer with total orders counted.

3. Final Answer:

   customer_id, total_orders -> Option A

4. Quick Check:

   Output columns = customer_id and total_orders [OK]

Hint: GROUP BY columns + aggregated columns appear in output [OK]

Common Mistakes:

• Expecting order_id as output column
• Missing aggregated column in output
• Confusing GROUP BY with WHERE clause

4. You wrote this dbt model in the marts layer:

SELECT
  customer_id,
  SUM(total_orders) AS total_orders
FROM {{ ref('int_customer_orders') }}
GROUP BY customer_id

But the model fails with an error about total_orders. What is the likely cause?

medium

A. Missing GROUP BY clause

B. SUM() cannot be used in marts layer

C. Incorrect use of {{ ref() }} syntax

D. The referenced model int_customer_orders does not have a column named total_orders

Solution

1. Step 1: Check referenced model columns

   The error suggests total_orders column is missing in int_customer_orders.

2. Step 2: Validate aggregation usage

   SUM() is valid in marts layer and GROUP BY is present, so those are not issues.

3. Final Answer:

   The referenced model int_customer_orders does not have a column named total_orders -> Option D

4. Quick Check:

   Column missing in referenced model causes error [OK]

Hint: Check referenced model columns before aggregating [OK]

Common Mistakes:

• Assuming SUM() is invalid in marts
• Forgetting GROUP BY with aggregation
• Misusing ref() syntax

5. You want to build a sales report in the marts layer that shows total sales per region. Your staging layer has raw sales data, and the intermediate layer calculates sales per store. Which is the best approach to build this report using the staging, intermediate, and marts pattern?

hard

A. Reference the intermediate sales per store model in the marts layer and aggregate sales by region

B. Directly query the raw sales data in the marts layer and group by region

C. Aggregate sales by region in the staging layer

D. Create the sales report in the intermediate layer without using marts

Solution

1. Step 1: Follow pattern responsibilities

   Staging cleans raw data, intermediate transforms data per store, marts create final aggregates like sales per region.

2. Step 2: Use intermediate model in marts

   Use the intermediate sales per store model as input to marts, then aggregate by region for the report.

3. Final Answer:

   Reference the intermediate sales per store model in the marts layer and aggregate sales by region -> Option A

4. Quick Check:

   Marts aggregate intermediate data for final reports [OK]

Hint: Build reports in marts using intermediate models [OK]

Common Mistakes:

• Skipping intermediate layer aggregation
• Aggregating raw data directly in marts
• Doing final aggregation in staging