Snowflake architecture (storage, compute, services layers) - Step-by-Step Execution

Process Flow - Snowflake architecture (storage, compute, services layers)

User Query

↓

Services Layer

↓

Compute Layer

↓

Storage Layer

↓

Return Data to User

User requests go through the Services layer, which manages tasks and security, then to Compute for processing, and Storage for data. Results flow back the same way.

Execution Sample

Snowflake

-- User runs a query
SELECT * FROM sales_data WHERE region = 'West';

This query is processed by Snowflake's architecture layers to return filtered sales data.

Process Table

Step	Layer	Action	Details	Result
1	Services Layer	Receive query	Parse and validate user query	Query accepted
2	Services Layer	Check permissions	Verify user access rights	Access granted
3	Compute Layer	Allocate warehouse	Assign virtual warehouse for query	Warehouse ready
4	Compute Layer	Process query	Translate query to execution plan	Execution plan created
5	Storage Layer	Fetch data	Retrieve sales_data from cloud storage	Data retrieved
6	Compute Layer	Filter data	Apply WHERE region = 'West'	Filtered data ready
7	Compute Layer	Return results	Send data back to Services layer	Results ready
8	Services Layer	Deliver results	Send data to user	User receives data
9	-	End	Query complete	Execution finished

💡 Query completes after results are delivered to the user

Status Tracker

Variable	Start	After Step 3	After Step 5	After Step 6	Final
Query	User input	Assigned to warehouse	Data fetched from storage	Data filtered by region	Results sent to user
Warehouse	None	Active	Active	Active	Released after query

Key Moments - 3 Insights

Why does the Services layer check permissions before processing?

The Services layer ensures security by verifying user rights before any compute or storage access, as shown in step 2 of the execution_table.

How does Compute layer handle the query without storing data?

Compute processes the query using data fetched from Storage but does not store data itself, demonstrated in steps 4, 6, and 7.

What happens if multiple users run queries at the same time?

Compute layer can allocate multiple virtual warehouses independently, allowing parallel processing without affecting Storage or Services layers.

Visual Quiz - 3 Questions

Look at the execution_table, at which step does the Compute layer filter data?

A Step 4

B Step 2

C Step 6

D Step 8

💡 Hint

Check the 'Layer' and 'Action' columns in execution_table rows

According to variable_tracker, what is the state of the Warehouse after Step 3?

A Inactive

B Active

C Released

D Not assigned

💡 Hint

Look at the 'Warehouse' row under 'After Step 3' column

If the Services layer denies access at Step 2, what happens next?

A Query is rejected and stops

B Query proceeds to Compute layer

C Data is fetched from Storage anyway

D Results are sent to user immediately

💡 Hint

Refer to the purpose of Step 2 in execution_table about permission checks

Concept Snapshot

Snowflake architecture has three layers:
1. Services Layer: Manages security, metadata, and query parsing.
2. Compute Layer: Runs queries using virtual warehouses.
3. Storage Layer: Stores all data centrally in cloud storage.
Layers work independently for scalability and performance.

Full Transcript

Snowflake architecture separates tasks into three layers. The Services layer receives and validates user queries, checking permissions to keep data secure. Then the Compute layer allocates virtual warehouses to process queries without storing data itself. The Storage layer holds all data in cloud storage, accessed by Compute when needed. This separation allows Snowflake to scale efficiently and handle many users at once. The execution flow starts with the user query entering Services, moving to Compute for processing, fetching data from Storage, and returning results back through Services to the user.

Practice

1. Which layer in Snowflake architecture is responsible for storing data securely in the cloud?

easy

A. Network layer

B. Storage layer

C. Services layer

D. Compute layer

Solution

1. Step 1: Understand Snowflake layers

   Snowflake architecture has three main layers: storage, compute, and services.

2. Step 2: Identify the storage role

   The storage layer holds all the data safely in the cloud, separate from compute and services.

3. Final Answer:

   Storage layer -> Option B

4. Quick Check:

   Storage = Data storage [OK]

Hint: Storage layer always holds your data safely [OK]

Common Mistakes:

• Confusing compute with storage
• Thinking services store data
• Selecting network layer which doesn't exist in Snowflake

2. Which Snowflake layer runs queries and can scale independently?

easy

A. Compute layer

B. Services layer

C. Storage layer

D. Security layer

Solution

1. Step 1: Recall Snowflake layers

   Snowflake separates compute, storage, and services layers.

2. Step 2: Identify compute layer role

   The compute layer runs queries and can scale up or down independently from storage.

3. Final Answer:

   Compute layer -> Option A

4. Quick Check:

   Compute = Runs queries [OK]

Hint: Compute layer runs queries and scales [OK]

Common Mistakes:

• Choosing storage for query execution
• Confusing services with compute
• Selecting security layer which is not a main layer

3. Given Snowflake's architecture, what happens if you pause a compute warehouse?

medium

A. Data in storage is deleted

B. Services layer shuts down

C. Compute and storage both pause

D. Queries stop running but data remains intact

Solution

1. Step 1: Understand compute warehouse pause

   Pausing compute stops query processing but does not affect stored data.

2. Step 2: Analyze impact on storage and services

   Storage remains active and data is safe; services continue managing metadata and security.

3. Final Answer:

   Queries stop running but data remains intact -> Option D

4. Quick Check:

   Pause compute = stop queries, keep data [OK]

Hint: Pausing compute stops queries, storage stays safe [OK]

Common Mistakes:

• Thinking data is deleted on pause
• Assuming services layer stops
• Believing storage also pauses

4. A user reports slow query performance. Which Snowflake layer should you check first for scaling issues?

medium

A. Compute layer

B. Storage layer

C. Services layer

D. Network layer

Solution

1. Step 1: Identify cause of slow queries

   Slow queries usually relate to compute resources being insufficient.

2. Step 2: Check compute layer scaling

   Compute layer runs queries and can be scaled up or out to improve performance.

3. Final Answer:

   Compute layer -> Option A

4. Quick Check:

   Slow queries? Check compute scaling [OK]

Hint: Slow queries? Scale compute layer first [OK]

Common Mistakes:

• Checking storage for query speed
• Blaming services layer for performance
• Selecting network layer which is not part of Snowflake

5. How does Snowflake's services layer contribute to security and metadata management?

hard

A. It stores all user data and query results

B. It runs queries and manages compute resources

C. It manages authentication, metadata, and transaction coordination

D. It handles physical storage of data files

Solution

1. Step 1: Understand services layer role

   The services layer manages security, metadata, and coordinates transactions.

2. Step 2: Differentiate from storage and compute

   Storage holds data; compute runs queries; services handle control tasks like authentication and metadata.

3. Final Answer:

   It manages authentication, metadata, and transaction coordination -> Option C

4. Quick Check:

   Services = Security + metadata + coordination [OK]

Hint: Services layer controls security and metadata [OK]

Common Mistakes:

• Thinking services store data
• Confusing compute with services
• Assuming services run queries