Multi-cluster warehouses in Snowflake - Step-by-Step Execution

Process Flow - Multi-cluster warehouses

User Query Arrives

↓

Check Warehouse Load

↓

Is Load High?

No→Use Single Cluster

Yes↓

Start Additional Clusters

↓

Distribute Queries Across Clusters

↓

Monitor Load and Scale Down

↓

Return Results to User

The system checks warehouse load and adds clusters if needed to handle more queries, then scales down when load decreases.

Execution Sample

Snowflake

CREATE WAREHOUSE my_wh
  WAREHOUSE_SIZE = 'MEDIUM'
  MIN_CLUSTER_COUNT = 1
  MAX_CLUSTER_COUNT = 3;

This code creates a multi-cluster warehouse that can scale between 1 and 3 clusters based on query load.

Process Table

Step	Warehouse Load	Clusters Active	Action Taken	Result
1	Low	1	Use single cluster	Queries processed by 1 cluster
2	Medium	1	Monitor load	No change, still 1 cluster
3	High	1	Start 2nd cluster	Queries distributed between 2 clusters
4	Very High	2	Start 3rd cluster	Queries distributed among 3 clusters
5	Medium	3	Scale down to 2 clusters	One cluster stopped, load balanced
6	Low	2	Scale down to 1 cluster	Back to single cluster
7	Low	1	Maintain single cluster	Stable state
8	Low	1	No further action	Execution ends

💡 Load is low and only 1 cluster active, no scaling needed

Status Tracker

Variable	Start	After Step 3	After Step 4	After Step 6	Final
Warehouse Load	Low	High	Very High	Low	Low
Clusters Active	1	2	3	1	1

Key Moments - 3 Insights

Why does the warehouse start additional clusters only when load is high?

Because as shown in execution_table steps 3 and 4, additional clusters start only when load reaches 'High' or 'Very High' to handle more queries efficiently.

What happens when the load decreases after scaling up?

As seen in steps 5 and 6, the warehouse scales down clusters to save resources, stopping extra clusters when load is medium or low.

Can the warehouse have zero clusters active?

No, the minimum cluster count is 1, so there is always at least one cluster active, as shown in all steps of the execution_table.

Visual Quiz - 3 Questions

Look at the execution_table, how many clusters are active at step 4?

A 1

B 2

C 3

D 4

💡 Hint

Refer to the 'Clusters Active' column at step 4 in the execution_table.

At which step does the warehouse scale down from 3 to 2 clusters?

A Step 3

B Step 5

C Step 6

D Step 7

💡 Hint

Check the 'Action Taken' column for scaling down actions in the execution_table.

If the maximum cluster count was set to 2, what would happen at step 4?

A Stay at 2 clusters and distribute queries

B Scale down to 1 cluster

C Start 3rd cluster anyway

D Stop all clusters

💡 Hint

Look at the 'MAX_CLUSTER_COUNT' setting in the execution_sample and relate it to cluster limits.

Concept Snapshot

Multi-cluster warehouses automatically add or remove compute clusters
based on query load to improve performance and concurrency.
Set MIN_CLUSTER_COUNT and MAX_CLUSTER_COUNT to control scaling.
Queries are distributed across active clusters.
Clusters scale down when load decreases to save costs.

Full Transcript

Multi-cluster warehouses in Snowflake help handle varying query loads by automatically adding or removing compute clusters. When query load is low, a single cluster processes queries. As load increases, additional clusters start to share the work, improving speed and concurrency. When load drops, extra clusters stop to save resources. This behavior is controlled by settings like MIN_CLUSTER_COUNT and MAX_CLUSTER_COUNT. The execution table shows step-by-step how clusters start and stop based on load changes, ensuring efficient resource use and fast query processing.

Practice

1. What is the main purpose of a multi-cluster warehouse in Snowflake?

easy

A. To automatically add or remove compute clusters based on workload demand

B. To store data in multiple regions for disaster recovery

C. To encrypt data at rest and in transit

D. To create backups of data automatically every hour

Solution

1. Step 1: Understand multi-cluster warehouse functionality

   Multi-cluster warehouses adjust the number of compute clusters automatically to handle workload changes.

2. Step 2: Compare options with this functionality

   Only To automatically add or remove compute clusters based on workload demand describes automatic scaling of compute clusters based on demand.

3. Final Answer:

   To automatically add or remove compute clusters based on workload demand -> Option A

4. Quick Check:

   Multi-cluster warehouses scale clusters automatically = A [OK]

Hint: Remember: multi-cluster means auto scaling clusters [OK]

Common Mistakes:

• Confusing multi-cluster warehouses with data replication
• Thinking multi-cluster warehouses handle backups
• Assuming multi-cluster warehouses manage encryption

2. Which of the following is the correct way to set the minimum and maximum cluster count for a multi-cluster warehouse in Snowflake?

easy

A. ALTER WAREHOUSE mywh SET MIN_CLUSTER_COUNT = 2, MAX_CLUSTER_COUNT = 5;

B. ALTER WAREHOUSE mywh SET MIN_CLUSTERS = 2, MAX_CLUSTERS = 5;

C. ALTER WAREHOUSE mywh SET MIN_CLUSTER = 2, MAX_CLUSTER = 5;

D. ALTER WAREHOUSE mywh SET CLUSTER_MIN = 2, CLUSTER_MAX = 5;

Solution

1. Step 1: Recall correct parameter names for cluster count

   The correct parameters are MIN_CLUSTERS and MAX_CLUSTERS.

2. Step 2: Match syntax with correct parameters

   ALTER WAREHOUSE mywh SET MIN_CLUSTERS = 2, MAX_CLUSTERS = 5; uses the exact correct syntax and parameter names for Snowflake.

3. Final Answer:

   ALTER WAREHOUSE mywh SET MIN_CLUSTERS = 2, MAX_CLUSTERS = 5; -> Option B

4. Quick Check:

   Use MIN_CLUSTERS and MAX_CLUSTERS for scaling [OK]

Hint: Use full parameter names with _CLUSTERS suffix [OK]

Common Mistakes:

• Using incorrect parameter names like MIN_CLUSTER_COUNT
• Omitting the _CLUSTERS suffix
• Using wrong syntax for ALTER WAREHOUSE

3. Consider a multi-cluster warehouse configured with MIN_CLUSTER_COUNT=1 and MAX_CLUSTER_COUNT=3. If the workload suddenly increases, what will happen?

medium

A. The warehouse will reduce clusters to save cost

B. The warehouse will stay at 1 cluster and queue queries

C. The warehouse will shut down automatically

D. The warehouse will add clusters up to 3 to handle the workload

Solution

1. Step 1: Understand scaling behavior of multi-cluster warehouses

   When workload increases, Snowflake adds clusters up to the MAX_CLUSTER_COUNT to reduce query wait times.

2. Step 2: Apply configuration limits

   Since MAX_CLUSTER_COUNT is 3, the warehouse can scale from 1 up to 3 clusters automatically.

3. Final Answer:

   The warehouse will add clusters up to 3 to handle the workload -> Option D

4. Quick Check:

   Workload increase triggers cluster scaling up to max [OK]

Hint: Scaling adds clusters up to max limit on demand [OK]

Common Mistakes:

• Assuming clusters stay fixed at minimum count
• Thinking warehouse shuts down on high load
• Believing clusters reduce during high workload

4. You configured a multi-cluster warehouse with MIN_CLUSTER_COUNT=2 and MAX_CLUSTER_COUNT=4, but it never scales beyond 2 clusters even under heavy load. What is the most likely cause?

medium

A. The warehouse is paused

B. The MAX_CLUSTER_COUNT parameter is set incorrectly

C. The warehouse is set to SINGLE cluster mode instead of MULTI cluster mode

D. The warehouse size is too small to scale

Solution

1. Step 1: Check warehouse mode setting

   Multi-cluster scaling only works if the warehouse is in MULTI cluster mode, not SINGLE cluster mode.

2. Step 2: Understand effect of SINGLE cluster mode

   If set to SINGLE cluster mode, the warehouse will not scale beyond one cluster regardless of MIN or MAX settings.

3. Final Answer:

   The warehouse is set to SINGLE cluster mode instead of MULTI cluster mode -> Option C

4. Quick Check:

   Multi-cluster scaling requires MULTI cluster mode [OK]

Hint: Check warehouse mode: must be MULTI cluster for scaling [OK]

Common Mistakes:

• Ignoring warehouse mode and expecting scaling
• Assuming warehouse size limits scaling
• Not realizing paused warehouse cannot scale

5. You want to optimize a multi-cluster warehouse for unpredictable workloads that spike quickly but also have long idle periods. Which configuration is best to balance performance and cost?

hard

A. Set MIN_CLUSTER_COUNT=1 and MAX_CLUSTER_COUNT=5 with auto-suspend enabled

B. Set MIN_CLUSTER_COUNT=5 and MAX_CLUSTER_COUNT=5 with auto-suspend disabled

C. Set MIN_CLUSTER_COUNT=3 and MAX_CLUSTER_COUNT=3 with auto-suspend enabled

D. Set MIN_CLUSTER_COUNT=0 and MAX_CLUSTER_COUNT=1 with auto-suspend disabled

Solution

1. Step 1: Analyze workload characteristics

   Unpredictable spikes require ability to scale up quickly; long idle times require cost saving by suspending.

2. Step 2: Evaluate configuration options

   Set MIN_CLUSTER_COUNT=1 and MAX_CLUSTER_COUNT=5 with auto-suspend enabled allows scaling from 1 to 5 clusters and enables auto-suspend to save cost during idle periods.

3. Step 3: Compare other options

   Configurations with fixed cluster counts reduce flexibility; MIN_CLUSTER_COUNT=0 and MAX_CLUSTER_COUNT=1 with auto-suspend disabled limits max clusters and increases cost.

4. Final Answer:

   Set MIN_CLUSTER_COUNT=1 and MAX_CLUSTER_COUNT=5 with auto-suspend enabled -> Option A

5. Quick Check:

   Flexible scaling plus auto-suspend balances cost and performance [OK]

Hint: Use low min, high max clusters with auto-suspend for spikes [OK]

Common Mistakes:

• Fixing cluster count too high wastes cost
• Disabling auto-suspend wastes resources during idle
• Setting max clusters too low limits performance