Why Snowflake separates compute from storage - See It in Action

Why Snowflake Separates Compute from Storage

📖 Scenario: Imagine you run a busy online store. You want your data system to be fast, flexible, and cost-effective. Snowflake is a cloud data platform that helps by separating compute power from storage space.

🎯 Goal: Build a simple Snowflake setup that shows how compute and storage are separate. You will create a storage area, a compute warehouse, and link them to see how they work independently.

📋 What You'll Learn

Create a database to hold data (storage)

Create a warehouse to process queries (compute)

Show how to use the warehouse to query data from the database

Demonstrate that storage and compute can scale separately

💡 Why This Matters

🌍 Real World

Many companies use Snowflake to store large amounts of data and run fast queries without paying for unused compute power.

💼 Career

Understanding compute and storage separation is key for cloud data engineers and architects working with Snowflake or similar cloud data platforms.

1

Create a database for storage

Write a Snowflake SQL command to create a database called store_data which will hold your data separately from compute.

Snowflake

-- Your code here to create the database

Hint

Use the CREATE DATABASE command with the exact name store_data.

2

Create a warehouse for compute

Write a Snowflake SQL command to create a warehouse called compute_wh that will handle query processing separately from storage.

Snowflake

CREATE DATABASE store_data;
-- Your code here to create the warehouse

Hint

Use CREATE WAREHOUSE with the name compute_wh. Set size to SMALL and enable auto suspend and resume.

3

Use the warehouse to query the database

Write Snowflake SQL commands to use the warehouse compute_wh and the database store_data. Then create a simple table products with columns id and name inside store_data.

Snowflake

CREATE DATABASE store_data;
CREATE WAREHOUSE compute_wh WITH WAREHOUSE_SIZE = 'SMALL' WAREHOUSE_TYPE = 'STANDARD' AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;
-- Your code here to use warehouse and database, then create table

Hint

Use USE WAREHOUSE and USE DATABASE commands. Then create the products table with id and name columns.

4

Show scaling compute and storage separately

Write Snowflake SQL commands to resize the warehouse compute_wh to MEDIUM without changing the database store_data. This shows compute scaling separately from storage.

Snowflake

CREATE DATABASE store_data;
CREATE WAREHOUSE compute_wh WITH WAREHOUSE_SIZE = 'SMALL' WAREHOUSE_TYPE = 'STANDARD' AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;
USE WAREHOUSE compute_wh;
USE DATABASE store_data;
CREATE TABLE products (id INT, name STRING);
-- Your code here to resize warehouse

Hint

Use ALTER WAREHOUSE to change the size of compute_wh to MEDIUM.

Practice

1. Why does Snowflake separate compute from storage?

easy

A. To combine compute and storage for faster processing

B. To store data only on local machines

C. To allow independent scaling of compute and storage resources

D. To limit the number of users accessing data

Solution

1. Step 1: Understand Snowflake's architecture

   Snowflake separates compute (processing power) and storage (data saved) so they can work independently.

2. Step 2: Identify the benefit of separation

   This separation allows users to scale compute resources up or down without affecting stored data, improving flexibility and cost.

3. Final Answer:

   To allow independent scaling of compute and storage resources -> Option C

4. Quick Check:

   Separation means independent scaling = A [OK]

Hint: Think: compute and storage can grow separately [OK]

Common Mistakes:

• Confusing separation with combining compute and storage
• Thinking data is stored only locally
• Believing separation limits user access

2. Which of the following is the correct way to describe Snowflake's compute and storage separation?

easy

A. Compute resources can be paused without affecting stored data

B. Storage automatically scales with compute usage

C. Compute and storage are tightly coupled in one system

D. Compute and storage must always scale together

Solution

1. Step 1: Review compute and storage behavior

   Snowflake allows compute (warehouses) to be paused or resized without impacting stored data.

2. Step 2: Match the correct description

   Compute resources can be paused without affecting stored data correctly states compute can be paused independently, which is a key feature.

3. Final Answer:

   Compute resources can be paused without affecting stored data -> Option A

4. Quick Check:

   Compute pause independent of storage = C [OK]

Hint: Remember: compute can pause, storage stays safe [OK]

Common Mistakes:

• Thinking compute and storage are tightly linked
• Assuming storage scales automatically with compute
• Believing compute and storage must scale together

3. Consider this scenario: You run multiple queries on Snowflake using different virtual warehouses. What is the main advantage of Snowflake's compute-storage separation in this case?

medium

A. Queries run slower because compute and storage are separate

B. Each warehouse can scale independently without data duplication

C. Data must be copied for each warehouse to run queries

D. Storage costs increase with each warehouse

Solution

1. Step 1: Analyze multiple warehouses running queries

   Snowflake allows multiple compute clusters (warehouses) to access the same storage without copying data.

2. Step 2: Understand the benefit of independent scaling

   Each warehouse can scale or pause independently, improving performance and cost without duplicating data.

3. Final Answer:

   Each warehouse can scale independently without data duplication -> Option B

4. Quick Check:

   Independent scaling, no data copy = D [OK]

Hint: Multiple warehouses share storage, no copies needed [OK]

Common Mistakes:

• Assuming data is copied for each warehouse
• Thinking compute-storage separation slows queries
• Believing storage costs rise with more warehouses

4. You notice your Snowflake compute warehouse is running slowly. You try to scale up compute but the performance does not improve. What could be a reason related to compute-storage separation?

medium

A. Compute and storage must be scaled together to improve speed

B. Compute warehouses cannot be resized after creation

C. Scaling compute automatically scales storage too

D. Storage is the bottleneck, not compute, since they are separate

Solution

1. Step 1: Understand compute-storage bottlenecks

   Since compute and storage are separate, scaling compute won't help if storage speed limits performance.

2. Step 2: Identify the correct reason

   Storage is the bottleneck, not compute, since they are separate correctly points out storage could be the bottleneck even if compute is scaled.

3. Final Answer:

   Storage is the bottleneck, not compute, since they are separate -> Option D

4. Quick Check:

   Separate storage bottleneck limits speed = B [OK]

Hint: Slow queries? Check storage bottleneck, not just compute [OK]

Common Mistakes:

• Assuming compute and storage scale together
• Believing compute cannot be resized
• Thinking scaling compute always fixes performance

5. You want to optimize costs and performance in Snowflake by using multiple virtual warehouses for different teams. How does Snowflake's separation of compute and storage help you achieve this?

hard

A. You can pause or resize warehouses independently while sharing the same data storage

B. You must create separate copies of data for each warehouse to avoid conflicts

C. Storage costs increase with each warehouse you create

D. Compute and storage are combined, so scaling one scales the other automatically

Solution

1. Step 1: Understand cost and performance optimization

   Using multiple warehouses allows teams to work independently without interfering with each other.

2. Step 2: Apply compute-storage separation benefits

   Since compute and storage are separate, warehouses can be paused or resized independently while sharing the same data, saving costs.

3. Final Answer:

   You can pause or resize warehouses independently while sharing the same data storage -> Option A

4. Quick Check:

   Independent warehouse control with shared storage = A [OK]

Hint: Pause or resize warehouses without copying data [OK]

Common Mistakes:

• Thinking data must be copied for each warehouse
• Assuming storage costs rise with more warehouses
• Believing compute and storage always scale together