Snowflake architecture (storage, compute, services layers) - Mini Project: Build & Apply

Snowflake Architecture: Storage, Compute, and Services Layers

📖 Scenario: You are working as a cloud architect for a company that wants to understand how Snowflake organizes its cloud data platform. Snowflake separates its architecture into three main layers: storage, compute, and services. This separation helps the company scale efficiently and manage data securely.

🎯 Goal: Build a simple Snowflake setup that demonstrates the separation of storage, compute, and services layers by creating a database, a warehouse, and a role with specific privileges. This will help you see how Snowflake manages data storage, query processing, and access control separately.

📋 What You'll Learn

Create a database named company_data to represent the storage layer

Create a warehouse named compute_wh to represent the compute layer

Create a role named data_analyst to represent the services layer

Grant the data_analyst role usage on the warehouse and select privileges on the database

💡 Why This Matters

🌍 Real World

Snowflake's architecture allows companies to store large amounts of data securely, process queries efficiently, and manage user access separately. This separation helps in scaling and optimizing costs.

💼 Career

Understanding Snowflake's architecture is essential for cloud architects, data engineers, and database administrators who design and manage cloud data platforms.

1

Create the storage layer database

Create a database called company_data to represent the storage layer in Snowflake.

Snowflake

# Create the database company_data
# Your code here

Hint

Use the CREATE DATABASE command followed by the database name.

2

Create the compute layer warehouse

Create a warehouse called compute_wh to represent the compute layer in Snowflake.

Snowflake

CREATE DATABASE company_data;
# Create the warehouse compute_wh
# Your code here

Hint

Use CREATE WAREHOUSE with a small size and enable auto suspend and resume for cost efficiency.

3

Create the services layer role

Create a role called data_analyst to represent the services layer in Snowflake.

Snowflake

CREATE DATABASE company_data;
CREATE WAREHOUSE compute_wh WITH WAREHOUSE_SIZE = 'XSMALL' WAREHOUSE_TYPE = 'STANDARD' AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;
# Create the role data_analyst
# Your code here

Hint

Use CREATE ROLE followed by the role name.

4

Grant privileges to the role

Grant the data_analyst role usage on the compute_wh warehouse and select privileges on the company_data database.

Snowflake

CREATE DATABASE company_data;
CREATE WAREHOUSE compute_wh WITH WAREHOUSE_SIZE = 'XSMALL' WAREHOUSE_TYPE = 'STANDARD' AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;
CREATE ROLE data_analyst;
# Grant privileges to data_analyst role
# Your code here

Hint

Use GRANT USAGE ON WAREHOUSE and GRANT SELECT ON DATABASE commands for the role.

Practice

1. Which layer in Snowflake architecture is responsible for storing data securely in the cloud?

easy

A. Network layer

B. Storage layer

C. Services layer

D. Compute layer

Solution

1. Step 1: Understand Snowflake layers

   Snowflake architecture has three main layers: storage, compute, and services.

2. Step 2: Identify the storage role

   The storage layer holds all the data safely in the cloud, separate from compute and services.

3. Final Answer:

   Storage layer -> Option B

4. Quick Check:

   Storage = Data storage [OK]

Hint: Storage layer always holds your data safely [OK]

Common Mistakes:

• Confusing compute with storage
• Thinking services store data
• Selecting network layer which doesn't exist in Snowflake

2. Which Snowflake layer runs queries and can scale independently?

easy

A. Compute layer

B. Services layer

C. Storage layer

D. Security layer

Solution

1. Step 1: Recall Snowflake layers

   Snowflake separates compute, storage, and services layers.

2. Step 2: Identify compute layer role

   The compute layer runs queries and can scale up or down independently from storage.

3. Final Answer:

   Compute layer -> Option A

4. Quick Check:

   Compute = Runs queries [OK]

Hint: Compute layer runs queries and scales [OK]

Common Mistakes:

• Choosing storage for query execution
• Confusing services with compute
• Selecting security layer which is not a main layer

3. Given Snowflake's architecture, what happens if you pause a compute warehouse?

medium

A. Data in storage is deleted

B. Services layer shuts down

C. Compute and storage both pause

D. Queries stop running but data remains intact

Solution

1. Step 1: Understand compute warehouse pause

   Pausing compute stops query processing but does not affect stored data.

2. Step 2: Analyze impact on storage and services

   Storage remains active and data is safe; services continue managing metadata and security.

3. Final Answer:

   Queries stop running but data remains intact -> Option D

4. Quick Check:

   Pause compute = stop queries, keep data [OK]

Hint: Pausing compute stops queries, storage stays safe [OK]

Common Mistakes:

• Thinking data is deleted on pause
• Assuming services layer stops
• Believing storage also pauses

4. A user reports slow query performance. Which Snowflake layer should you check first for scaling issues?

medium

A. Compute layer

B. Storage layer

C. Services layer

D. Network layer

Solution

1. Step 1: Identify cause of slow queries

   Slow queries usually relate to compute resources being insufficient.

2. Step 2: Check compute layer scaling

   Compute layer runs queries and can be scaled up or out to improve performance.

3. Final Answer:

   Compute layer -> Option A

4. Quick Check:

   Slow queries? Check compute scaling [OK]

Hint: Slow queries? Scale compute layer first [OK]

Common Mistakes:

• Checking storage for query speed
• Blaming services layer for performance
• Selecting network layer which is not part of Snowflake

5. How does Snowflake's services layer contribute to security and metadata management?

hard

A. It stores all user data and query results

B. It runs queries and manages compute resources

C. It manages authentication, metadata, and transaction coordination

D. It handles physical storage of data files

Solution

1. Step 1: Understand services layer role

   The services layer manages security, metadata, and coordinates transactions.

2. Step 2: Differentiate from storage and compute

   Storage holds data; compute runs queries; services handle control tasks like authentication and metadata.

3. Final Answer:

   It manages authentication, metadata, and transaction coordination -> Option C

4. Quick Check:

   Services = Security + metadata + coordination [OK]

Hint: Services layer controls security and metadata [OK]

Common Mistakes:

• Thinking services store data
• Confusing compute with services
• Assuming services run queries