Why advanced patterns solve complex analytics in dbt - See It in Action

Why advanced patterns solve complex analytics

📖 Scenario: You work as a data analyst in a retail company. You have a large sales dataset and want to analyze customer buying patterns over time. Simple queries are slow and hard to maintain. Your manager asks you to use advanced dbt patterns to build efficient, reusable models that solve complex analytics problems.

🎯 Goal: Build a dbt project that uses advanced patterns like CTEs, incremental models, and macros to analyze customer purchase frequency and segment customers by activity level.

📋 What You'll Learn

Create a base model with raw sales data

Add a config variable to enable incremental loading

Use a CTE and window functions to calculate purchase frequency per customer

Print the final customer segments with counts

💡 Why This Matters

🌍 Real World

Retail companies analyze customer purchase patterns to target marketing and improve sales.

💼 Career

Data analysts and engineers use dbt advanced patterns to build scalable, maintainable analytics pipelines.

1

Create the base sales model

Create a dbt model file called base_sales.sql with a SELECT statement that selects customer_id, order_id, and order_date from the raw_sales table.

dbt

-- Your code here

Hint

This base model selects the raw sales data columns needed for analysis.

2

Add incremental model configuration

In the base_sales.sql model, add a config block to enable incremental loading with unique_key='order_id'.

dbt

select customer_id, order_id, order_date from raw_sales

-- Add config block for incremental model here

Hint

Use the config Jinja function at the top of the model to set incremental materialization.

3

Calculate purchase frequency using CTE and window functions

Create a new model called customer_frequency.sql. Use a CTE named base that selects all columns from ref('base_sales'). Then, in the main query, use a window function COUNT(order_id) OVER (PARTITION BY customer_id) to calculate purchase_count per customer.

dbt

-- Your code here

Hint

Use a CTE to reference the base model and a window function to count orders per customer.

4

Segment customers and output results

In the customer_frequency.sql model, add a final SELECT that uses a CASE statement to create a segment column: 'High' if purchase_count >= 5, 'Medium' if between 2 and 4, else 'Low'. Then, group by segment and count distinct customer_id. Print the results.

dbt

with base as (
    select * from {{ ref('base_sales') }}
)

-- Add segmentation and aggregation here

# Your code here

Hint

Use CASE to segment customers and GROUP BY to count customers per segment.

Practice

1. Why do advanced patterns in dbt help solve complex analytics problems?

easy

A. They break down complex tasks into smaller, manageable steps.

B. They make the code longer and harder to read.

C. They avoid using ref() and CTEs.

D. They remove the need for testing data models.

Solution

1. Step 1: Understand the role of advanced patterns

   Advanced patterns help by dividing complex analytics into smaller parts, making them easier to handle.

2. Step 2: Recognize the benefits in dbt

   Using features like ref() and CTEs supports this breakdown, improving clarity and maintainability.

3. Final Answer:

   They break down complex tasks into smaller, manageable steps. -> Option A

4. Quick Check:

   Advanced patterns simplify complexity = C [OK]

Hint: Think: smaller steps make big tasks easier [OK]

Common Mistakes:

• Assuming advanced patterns make code harder
• Ignoring the use of ref() in dbt
• Believing testing is not needed with advanced patterns

2. Which of the following is the correct way to reference another model in dbt using advanced patterns?

easy

A. SELECT * FROM ref('model_name')

B. SELECT * FROM model_name()

C. SELECT * FROM {{ ref('model_name') }}

D. SELECT * FROM {{ model('model_name') }}

Solution

1. Step 1: Recall dbt syntax for referencing models

   In dbt, the correct way to reference another model is using the Jinja function {{ ref('model_name') }}.

2. Step 2: Identify the correct SQL syntax

   The reference must be inside double curly braces and used as a table name in the FROM clause.

3. Final Answer:

   SELECT * FROM {{ ref('model_name') }} -> Option C

4. Quick Check:

   Use {{ ref() }} with curly braces = B [OK]

Hint: Remember: always use {{ ref('model') }} in dbt SQL [OK]

Common Mistakes:

• Omitting curly braces around ref()
• Using parentheses like a function call without {{ }}
• Confusing ref() with model()

3. Given this dbt model SQL code using CTEs, what will be the final output table's column total_sales?

WITH sales AS (
  SELECT customer_id, amount
  FROM {{ ref('raw_sales') }}
),
filtered_sales AS (
  SELECT customer_id, amount
  FROM sales
  WHERE amount > 100
)
SELECT customer_id, SUM(amount) AS total_sales
FROM filtered_sales
GROUP BY customer_id

medium

A. Sum of all sales amounts per customer including amounts <= 100

B. Sum of sales amounts per customer where each amount is greater than 100

C. Count of sales transactions per customer

D. Average sales amount per customer

Solution

1. Step 1: Analyze the filtering condition in filtered_sales CTE

   The CTE filters sales to only include rows where amount > 100.

2. Step 2: Understand the aggregation in the final SELECT

   The final query sums the filtered amounts per customer, so total_sales is sum of amounts > 100 per customer.

3. Final Answer:

   Sum of sales amounts per customer where each amount is greater than 100 -> Option B

4. Quick Check:

   Filter then sum = D [OK]

Hint: Filter first, then aggregate sums [OK]

Common Mistakes:

• Ignoring the WHERE filter on amount
• Confusing sum with count or average
• Assuming all sales are included

4. You wrote this dbt model but it fails with a compilation error. What is the likely cause?

WITH base AS (
  SELECT *
  FROM ref('orders')
)
SELECT * FROM base

medium

A. Missing curly braces around ref function

B. Incorrect CTE name 'base'

C. SELECT * is not allowed in dbt models

D. Missing GROUP BY clause

Solution

1. Step 1: Check the usage of ref() in the CTE

   The ref function must be wrapped in double curly braces like {{ ref('orders') }} to be recognized by dbt.

2. Step 2: Identify the error cause

   Without curly braces, dbt treats ref('orders') as plain text, causing a compilation error.

3. Final Answer:

   Missing curly braces around ref function -> Option A

4. Quick Check:

   Always use {{ ref() }} with curly braces = A [OK]

Hint: Always wrap ref() in {{ }} in dbt SQL [OK]

Common Mistakes:

• Forgetting curly braces around ref()
• Thinking CTE names cause errors
• Assuming SELECT * is invalid in dbt

5. You want to create a dbt model that calculates the average order amount per customer but only for customers with more than 5 orders. Which advanced pattern best helps you write this cleanly and efficiently?

hard

A. Calculate average without filtering orders count

B. Write a single SELECT with multiple JOINs without CTEs

C. Use multiple separate models without referencing each other

D. Use nested CTEs: one to count orders per customer, another to filter, then calculate average

Solution

1. Step 1: Understand the problem requirements

   You need to filter customers by order count > 5, then calculate average order amount.

2. Step 2: Identify the best advanced pattern

   Using nested CTEs breaks the problem into steps: count orders, filter customers, then compute average, making code clear and maintainable.

3. Final Answer:

   Use nested CTEs: one to count orders per customer, another to filter, then calculate average -> Option D

4. Quick Check:

   Break complex tasks into steps with CTEs = A [OK]

Hint: Use nested CTEs to split filtering and aggregation [OK]

Common Mistakes:

• Trying to do all in one complex query
• Not filtering customers before averaging
• Avoiding model references and CTEs