Why Snowflake separates compute from storage - Why It Works

Introduction

Snowflake separates compute from storage to let you use each one independently. This means you can store lots of data without paying for compute power all the time. And you can run many queries at once without slowing down.

When you want to store large amounts of data but only run queries sometimes.

When multiple teams need to run queries on the same data without waiting for each other.

When you want to save money by scaling compute power up or down separately from storage.

When you want to keep your data safe and accessible even if compute resources change.

When you want fast query performance without copying or moving data.

Commands

This command creates a compute warehouse in Snowflake. The warehouse handles query processing separately from storage. Auto suspend and resume save costs by stopping compute when idle.

Terminal

CREATE WAREHOUSE my_warehouse WITH WAREHOUSE_SIZE = 'SMALL' WAREHOUSE_TYPE = 'STANDARD' AUTO_SUSPEND = 60 AUTO_RESUME = TRUE;

Expected Output

Statement executed successfully.

→

WAREHOUSE_SIZE - Sets the size of the compute warehouse to control processing power.

→

AUTO_SUSPEND - Automatically suspends the warehouse after 60 seconds of inactivity to save costs.

→

AUTO_RESUME - Automatically resumes the warehouse when a query is submitted.

This command tells Snowflake to use the created warehouse for running queries. It connects compute resources to your session.

Terminal

USE WAREHOUSE my_warehouse;

Expected Output

Statement executed successfully.

This command checks which warehouse is currently active for your session, confirming compute is separate and in use.

Terminal

SELECT CURRENT_WAREHOUSE();

Expected Output

CURRENT_WAREHOUSE()

------------------

MY_WAREHOUSE

This command lists all warehouses and their status, showing compute resources separately from storage.

Terminal

SHOW WAREHOUSES;

Expected Output

name          | state   | size  | type     | auto_suspend | auto_resume

MY_WAREHOUSE  | RUNNING | SMALL | STANDARD | 60           | TRUE

Key Concept

If you remember nothing else from this pattern, remember: Snowflake separates compute and storage so you can scale and pay for each independently.

Common Mistakes

Trying to run queries without selecting a warehouse first.

Queries need compute resources to run, and without selecting a warehouse, Snowflake cannot process them.

Always run 'USE WAREHOUSE your_warehouse;' before running queries.

Creating a warehouse without auto suspend enabled.

This causes compute to run continuously, increasing costs unnecessarily.

Set AUTO_SUSPEND to a reasonable time like 60 seconds to save costs.

Summary

Create a warehouse to provide compute power separately from storage.

Use the warehouse to run queries without affecting stored data.

Auto suspend and resume help save costs by managing compute usage automatically.

Practice

1. Why does Snowflake separate compute from storage?

easy

A. To combine compute and storage for faster processing

B. To store data only on local machines

C. To allow independent scaling of compute and storage resources

D. To limit the number of users accessing data

Solution

1. Step 1: Understand Snowflake's architecture

   Snowflake separates compute (processing power) and storage (data saved) so they can work independently.

2. Step 2: Identify the benefit of separation

   This separation allows users to scale compute resources up or down without affecting stored data, improving flexibility and cost.

3. Final Answer:

   To allow independent scaling of compute and storage resources -> Option C

4. Quick Check:

   Separation means independent scaling = A [OK]

Hint: Think: compute and storage can grow separately [OK]

Common Mistakes:

• Confusing separation with combining compute and storage
• Thinking data is stored only locally
• Believing separation limits user access

2. Which of the following is the correct way to describe Snowflake's compute and storage separation?

easy

A. Compute resources can be paused without affecting stored data

B. Storage automatically scales with compute usage

C. Compute and storage are tightly coupled in one system

D. Compute and storage must always scale together

Solution

1. Step 1: Review compute and storage behavior

   Snowflake allows compute (warehouses) to be paused or resized without impacting stored data.

2. Step 2: Match the correct description

   Compute resources can be paused without affecting stored data correctly states compute can be paused independently, which is a key feature.

3. Final Answer:

   Compute resources can be paused without affecting stored data -> Option A

4. Quick Check:

   Compute pause independent of storage = C [OK]

Hint: Remember: compute can pause, storage stays safe [OK]

Common Mistakes:

• Thinking compute and storage are tightly linked
• Assuming storage scales automatically with compute
• Believing compute and storage must scale together

3. Consider this scenario: You run multiple queries on Snowflake using different virtual warehouses. What is the main advantage of Snowflake's compute-storage separation in this case?

medium

A. Queries run slower because compute and storage are separate

B. Each warehouse can scale independently without data duplication

C. Data must be copied for each warehouse to run queries

D. Storage costs increase with each warehouse

Solution

1. Step 1: Analyze multiple warehouses running queries

   Snowflake allows multiple compute clusters (warehouses) to access the same storage without copying data.

2. Step 2: Understand the benefit of independent scaling

   Each warehouse can scale or pause independently, improving performance and cost without duplicating data.

3. Final Answer:

   Each warehouse can scale independently without data duplication -> Option B

4. Quick Check:

   Independent scaling, no data copy = D [OK]

Hint: Multiple warehouses share storage, no copies needed [OK]

Common Mistakes:

• Assuming data is copied for each warehouse
• Thinking compute-storage separation slows queries
• Believing storage costs rise with more warehouses

4. You notice your Snowflake compute warehouse is running slowly. You try to scale up compute but the performance does not improve. What could be a reason related to compute-storage separation?

medium

A. Compute and storage must be scaled together to improve speed

B. Compute warehouses cannot be resized after creation

C. Scaling compute automatically scales storage too

D. Storage is the bottleneck, not compute, since they are separate

Solution

1. Step 1: Understand compute-storage bottlenecks

   Since compute and storage are separate, scaling compute won't help if storage speed limits performance.

2. Step 2: Identify the correct reason

   Storage is the bottleneck, not compute, since they are separate correctly points out storage could be the bottleneck even if compute is scaled.

3. Final Answer:

   Storage is the bottleneck, not compute, since they are separate -> Option D

4. Quick Check:

   Separate storage bottleneck limits speed = B [OK]

Hint: Slow queries? Check storage bottleneck, not just compute [OK]

Common Mistakes:

• Assuming compute and storage scale together
• Believing compute cannot be resized
• Thinking scaling compute always fixes performance

5. You want to optimize costs and performance in Snowflake by using multiple virtual warehouses for different teams. How does Snowflake's separation of compute and storage help you achieve this?

hard

A. You can pause or resize warehouses independently while sharing the same data storage

B. You must create separate copies of data for each warehouse to avoid conflicts

C. Storage costs increase with each warehouse you create

D. Compute and storage are combined, so scaling one scales the other automatically

Solution

1. Step 1: Understand cost and performance optimization

   Using multiple warehouses allows teams to work independently without interfering with each other.

2. Step 2: Apply compute-storage separation benefits

   Since compute and storage are separate, warehouses can be paused or resized independently while sharing the same data, saving costs.

3. Final Answer:

   You can pause or resize warehouses independently while sharing the same data storage -> Option A

4. Quick Check:

   Independent warehouse control with shared storage = A [OK]

Hint: Pause or resize warehouses without copying data [OK]

Common Mistakes:

• Thinking data must be copied for each warehouse
• Assuming storage costs rise with more warehouses
• Believing compute and storage always scale together