Data mesh patterns with dbt - Step-by-Step Execution

Concept Flow - Data mesh patterns with dbt

Define Domain Data Models

↓

Create dbt Models per Domain

↓

Apply Transformations & Tests

↓

Publish Domain Data Products

↓

Consume Data Products Across Domains

↓

Monitor & Evolve Mesh

This flow shows how data mesh patterns use dbt to build domain-specific data models, transform and test them, publish as data products, and share across domains.

Execution Sample

dbt

with sales as (
  select * from raw.sales_data
),
customer_summary as (
  select customer_id, sum(amount) as total_spent
  from sales
  group by customer_id
)
select * from customer_summary

This dbt model transforms raw sales data into a customer summary showing total spent per customer.

Execution Table

Step	Action	SQL Query Part	Result Preview
1	Read raw sales data	select * from raw.sales_data	Rows of raw sales records
2	Aggregate sales by customer	select customer_id, sum(amount) as total_spent from sales group by customer_id	Customer IDs with total spent sums
3	Output final customer summary	select * from customer_summary	Table with customer_id and total_spent columns
4	End of model execution	N/A	Data ready for downstream consumption

💡 All transformations complete, data product ready for use

Variable Tracker

Variable	Start	After Step 1	After Step 2	Final
sales	undefined	raw sales data rows loaded	unchanged	unchanged
customer_summary	undefined	undefined	aggregated sums per customer	final output table

Key Moments - 2 Insights

Why do we create separate models for each domain in dbt?

Creating separate models per domain helps isolate data logic and ownership, as shown in the flow from defining domain models to publishing data products (see concept_flow). This makes data easier to manage and share.

How does dbt ensure data quality in data mesh?

dbt applies tests and transformations in each model step (see execution_table steps 2 and 3), catching errors early and ensuring reliable data products.

Visual Quiz - 3 Questions

Look at the execution_table, what is the result after Step 2?

A Raw sales data rows

B Customer IDs with total spent sums

C Final output table with all columns

D Empty table

💡 Hint

Refer to execution_table row 2 under 'Result Preview' for the aggregated result

At which step does the model output the final customer summary?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Check execution_table row 3 for the final output action

If we add a test to check for null customer_id, where in the flow would it fit?

A During transformations and tests step

B After publishing data products

C Before defining domain data models

D During data consumption across domains

💡 Hint

Look at concept_flow step 'Apply Transformations & Tests' for when quality checks happen

Concept Snapshot

Data mesh with dbt:
- Define domain-specific models
- Use dbt to transform and test data
- Publish as reusable data products
- Share across domains
- Monitor and evolve continuously

Full Transcript

Data mesh patterns with dbt involve creating domain-specific data models using dbt. The process starts by defining data models for each domain. Then, dbt runs SQL transformations and tests to prepare clean, reliable data products. These products are published and shared across domains for easy consumption. Monitoring and evolving the mesh ensures data stays accurate and useful. The example code shows how raw sales data is transformed into a customer summary with total spending. The execution table traces each step from reading raw data to outputting the final summary. Variables like 'sales' and 'customer_summary' change as data flows through the steps. Key moments clarify why domain separation and testing are important. The quiz checks understanding of the execution steps and where tests fit in the flow.