Staging, intermediate, and marts pattern in dbt - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the staging layer in the staging, intermediate, and marts pattern?

The staging layer is where raw data from source systems is cleaned and organized into a consistent format. It acts like a preparation area before deeper transformations.

intermediate

Describe the role of the intermediate layer in this pattern.

The intermediate layer applies business logic and combines data from staging tables. It prepares data for final use by cleaning, filtering, and joining datasets.

beginner

What is a data mart in the staging, intermediate, and marts pattern?

A data mart is the final layer where data is organized for easy access by business users. It contains summarized, cleaned, and ready-to-use data for reporting and analysis.

intermediate

Why is it helpful to separate data processing into staging, intermediate, and marts layers?

Separating layers helps keep data organized, makes debugging easier, and allows teams to work on different parts independently. It also improves data quality and performance.

intermediate

In dbt, how do models typically correspond to the staging, intermediate, and marts layers?

In dbt, models are grouped by folders or tags to represent each layer: staging models load raw data, intermediate models transform and combine data, and marts models prepare final datasets for analysis.

What is the main job of the staging layer?

A Clean and organize raw source data

B Create final reports

C Visualize data

D Archive old data

The staging layer cleans and organizes raw data from sources before further processing.

Which layer applies business logic and combines data?

A Staging

B Intermediate

C Marts

D Source

The intermediate layer applies business rules and combines data from staging.

What is stored in the data marts layer?

A Final cleaned and summarized data

B Temporary files

C Raw data

D Backup data

Data marts contain final, cleaned, and summarized data ready for analysis.

Why separate data processing into layers?

A To avoid using dbt

B To confuse users

C To slow down processing

D To organize data and improve quality

Layering helps organize data, improve quality, and makes debugging easier.

In dbt, how are models organized to reflect these layers?

A By file size

B Randomly

C By folder or tags for staging, intermediate, and marts

D By color

dbt models are grouped by folders or tags to represent each layer clearly.

Explain the purpose of each layer in the staging, intermediate, and marts pattern.

Think about how data flows from raw to ready-to-use.

Describe how using these layers benefits a data project.

Consider the advantages of clear separation in work.

Practice

1. What is the main purpose of the staging layer in the dbt pattern Staging, intermediate, and marts?

easy

A. To create final reports and dashboards

B. To aggregate data for business metrics

C. To clean and prepare raw data for further transformations

D. To store backup copies of raw data

Solution

1. Step 1: Understand the role of staging

   The staging layer is where raw data is cleaned and standardized to make it ready for analysis.

2. Step 2: Differentiate from other layers

   Intermediate and marts layers perform transformations and aggregations, while staging focuses on cleaning.

3. Final Answer:

   To clean and prepare raw data for further transformations -> Option C

4. Quick Check:

   Staging = cleaning raw data [OK]

Hint: Staging = clean raw data first [OK]

Common Mistakes:

• Confusing staging with marts layer
• Thinking staging creates final reports
• Assuming staging stores backups

2. Which of the following is the correct way to reference a model in dbt within the intermediate layer?

easy

A. SELECT * FROM stg_customers

B. SELECT * FROM {{ ref('stg_customers') }}

C. SELECT * FROM ref('stg_customers')

D. SELECT * FROM {{ source('stg_customers') }}

Solution

1. Step 1: Recall dbt model referencing syntax

   dbt uses the Jinja function ref() inside double curly braces to reference models.

2. Step 2: Identify correct syntax

   The correct syntax is {{ ref('model_name') }}, so {{ ref('stg_customers') }} is correct.

3. Final Answer:

   SELECT * FROM {{ ref('stg_customers') }} -> Option B

4. Quick Check:

   Use {{ ref('model') }} to reference models [OK]

Hint: Use {{ ref('model_name') }} to reference models [OK]

Common Mistakes:

• Omitting curly braces around ref()
• Using source() instead of ref() for models
• Calling ref() without curly braces

3. Given the following dbt model code in the intermediate layer:

SELECT
  customer_id,
  COUNT(order_id) AS total_orders
FROM {{ ref('stg_orders') }}
GROUP BY customer_id

What will be the output columns of this model?

medium

A. customer_id, total_orders

B. order_id, total_orders

C. customer_id, order_id

D. customer_id only

Solution

1. Step 1: Analyze SELECT statement

   The query selects customer_id and counts order_id as total_orders.

2. Step 2: Understand GROUP BY effect

   Grouping by customer_id means output will have one row per customer with total orders counted.

3. Final Answer:

   customer_id, total_orders -> Option A

4. Quick Check:

   Output columns = customer_id and total_orders [OK]

Hint: GROUP BY columns + aggregated columns appear in output [OK]

Common Mistakes:

• Expecting order_id as output column
• Missing aggregated column in output
• Confusing GROUP BY with WHERE clause

4. You wrote this dbt model in the marts layer:

SELECT
  customer_id,
  SUM(total_orders) AS total_orders
FROM {{ ref('int_customer_orders') }}
GROUP BY customer_id

But the model fails with an error about total_orders. What is the likely cause?

medium

A. Missing GROUP BY clause

B. SUM() cannot be used in marts layer

C. Incorrect use of {{ ref() }} syntax

D. The referenced model int_customer_orders does not have a column named total_orders

Solution

1. Step 1: Check referenced model columns

   The error suggests total_orders column is missing in int_customer_orders.

2. Step 2: Validate aggregation usage

   SUM() is valid in marts layer and GROUP BY is present, so those are not issues.

3. Final Answer:

   The referenced model int_customer_orders does not have a column named total_orders -> Option D

4. Quick Check:

   Column missing in referenced model causes error [OK]

Hint: Check referenced model columns before aggregating [OK]

Common Mistakes:

• Assuming SUM() is invalid in marts
• Forgetting GROUP BY with aggregation
• Misusing ref() syntax

5. You want to build a sales report in the marts layer that shows total sales per region. Your staging layer has raw sales data, and the intermediate layer calculates sales per store. Which is the best approach to build this report using the staging, intermediate, and marts pattern?

hard

A. Reference the intermediate sales per store model in the marts layer and aggregate sales by region

B. Directly query the raw sales data in the marts layer and group by region

C. Aggregate sales by region in the staging layer

D. Create the sales report in the intermediate layer without using marts

Solution

1. Step 1: Follow pattern responsibilities

   Staging cleans raw data, intermediate transforms data per store, marts create final aggregates like sales per region.

2. Step 2: Use intermediate model in marts

   Use the intermediate sales per store model as input to marts, then aggregate by region for the report.

3. Final Answer:

   Reference the intermediate sales per store model in the marts layer and aggregate sales by region -> Option A

4. Quick Check:

   Marts aggregate intermediate data for final reports [OK]

Hint: Build reports in marts using intermediate models [OK]

Common Mistakes:

• Skipping intermediate layer aggregation
• Aggregating raw data directly in marts
• Doing final aggregation in staging