Why Snowpark brings code to the data in Snowflake - Visual Breakdown

Process Flow - Why Snowpark brings code to the data

User writes code locally

↓

Code sent to Snowflake platform

↓

Code runs inside Snowflake near data

↓

Results sent back to user

↓

User gets output quickly and securely

This flow shows how Snowpark moves your code to where the data lives inside Snowflake, instead of moving data to your code.

Execution Sample

Snowflake

session.sql("SELECT * FROM sales WHERE amount > 100").collect()

This code runs inside Snowflake, filtering data close to where it is stored.

Process Table

Step	Action	Location	Data Movement	Result
1	User writes Snowpark code	User machine	No data moved	Code ready to send
2	Code sent to Snowflake	Network	Code moves, data stays	Code received by Snowflake
3	Code executes near data	Snowflake server	No data movement needed	Filtered data prepared
4	Results sent back	Network	Only filtered results move	User receives output
5	User processes results	User machine	No extra data movement	Final output ready

💡 Execution stops after results are returned to user, minimizing data transfer.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
Code	Local script	Sent to Snowflake	Executing	Executed	Complete
Data	In Snowflake	In Snowflake	Filtered in Snowflake	Filtered results sent	User received results

Key Moments - 3 Insights

Why doesn't Snowpark move all data to the user's machine?

Because moving large data is slow and costly. Snowpark runs code inside Snowflake where data lives, so only small results move back, as shown in execution_table rows 2 and 3.

How does running code near data improve speed?

It avoids delays from transferring big data over the network. execution_table row 3 shows code running inside Snowflake, filtering data before sending results.

What moves over the network when using Snowpark?

Only the code and the filtered results move, not the entire dataset. This is clear in execution_table rows 2 and 4.

Visual Quiz - 3 Questions

Look at the execution_table, at which step does the code start running near the data?

A Step 1

B Step 2

C Step 3

D Step 4

💡 Hint

Check the 'Location' and 'Action' columns in execution_table row 3.

According to variable_tracker, what happens to the data after step 3?

A Data moves to user machine

B Data is filtered inside Snowflake

C Data is deleted

D Data is copied to another server

💡 Hint

Look at the 'Data' row under 'After Step 3' in variable_tracker.

If Snowpark moved all data to the user machine, which execution_table step would change?

A Step 4

B Step 3

C Step 2

D Step 5

💡 Hint

Consider where data moves over the network in execution_table, especially step 4.

Concept Snapshot

Snowpark sends your code to Snowflake where data lives.
Code runs inside Snowflake, close to data.
Only filtered results move back to you.
This reduces data transfer and speeds up processing.
It keeps data secure and efficient.

Full Transcript

Snowpark works by sending your code to run inside Snowflake, where the data is stored. This means the data does not move to your computer. Instead, your code runs near the data, filtering or processing it there. Then, only the small results are sent back to you. This approach saves time and network resources, making data processing faster and safer. The execution flow starts with you writing code, sending it to Snowflake, running it near the data, and finally receiving the results.

Practice

1. What is the main benefit of Snowpark bringing code to the data inside Snowflake?

easy

A. It increases data duplication across systems.

B. It requires exporting data to external servers.

C. It only supports SQL queries.

D. It reduces data movement and improves performance.

Solution

1. Step 1: Understand Snowpark's design

   Snowpark runs code inside Snowflake, close to the data storage.

2. Step 2: Identify benefits of running code near data

   This reduces the need to move large data sets, saving time and resources.

3. Final Answer:

   It reduces data movement and improves performance. -> Option D

4. Quick Check:

   Code near data = less data movement [OK]

Hint: Code runs inside Snowflake, so data stays put [OK]

Common Mistakes:

• Thinking data must be exported to run code
• Believing Snowpark only supports SQL
• Assuming data is duplicated across systems

2. Which of the following is the correct way to create a Snowpark session in Python?

easy

A. session = SnowflakeSession.create(connection_parameters)

B. session = snowpark.connect(connection_parameters)

C. session = snowflake.snowpark.Session.builder.configs(connection_parameters).create()

D. session = snowflake.connect_snowpark(connection_parameters)

Solution

1. Step 1: Recall Snowpark Python session creation syntax

   Snowpark uses Session.builder.configs(...).create() to start a session.

2. Step 2: Match the correct method call

   session = snowflake.snowpark.Session.builder.configs(connection_parameters).create() matches the official syntax for creating a session in Snowpark Python API.

3. Final Answer:

   session = snowflake.snowpark.Session.builder.configs(connection_parameters).create() -> Option C

4. Quick Check:

   Session.builder.configs(...).create() = correct syntax [OK]

Hint: Look for builder.configs(...).create() pattern [OK]

Common Mistakes:

• Using incorrect method names like connect or connect_snowpark
• Missing builder or configs in the call
• Confusing Snowpark session with generic Snowflake connection

3. Given this Snowpark Python code snippet:

df = session.table('EMPLOYEES')
filtered = df.filter(df['DEPARTMENT'] == 'SALES')
count = filtered.count()

What does count represent?

medium

A. The number of rows where DEPARTMENT is 'SALES'.

B. The number of columns in the EMPLOYEES table.

C. The total number of rows in the EMPLOYEES table.

D. The number of distinct departments in EMPLOYEES.

Solution

1. Step 1: Understand the filter operation

   The code filters rows where DEPARTMENT equals 'SALES'.

2. Step 2: Understand the count method

   Count returns the number of rows in the filtered DataFrame.

3. Final Answer:

   The number of rows where DEPARTMENT is 'SALES'. -> Option A

4. Quick Check:

   filter + count = rows matching condition [OK]

Hint: Filter then count means count matching rows [OK]

Common Mistakes:

• Thinking count returns total rows without filter
• Confusing count with number of columns
• Assuming count returns distinct values

4. You wrote this Snowpark code but get an error:

df = session.table('EMPLOYEES')
filtered = df.filter(df.DEPARTMENT = 'SALES')
count = filtered.count()

What is the error and how to fix it?

medium

A. Add parentheses around the filter condition.

B. Use '==' for comparison instead of '=' in the filter condition.

C. Rename 'DEPARTMENT' column to lowercase.

D. Change 'filter' to 'where' method.

Solution

1. Step 1: Identify the filter condition syntax

   The filter uses '=' which is assignment, not comparison.

2. Step 2: Correct the comparison operator

   Use '==' to compare column values in Snowpark expressions.

3. Final Answer:

   Use '==' for comparison instead of '=' in the filter condition. -> Option B

4. Quick Check:

   Comparison needs '==' not '=' [OK]

Hint: Use '==' for equality checks in filters [OK]

Common Mistakes:

• Using single '=' instead of '==' in conditions
• Thinking 'where' is required instead of 'filter'
• Assuming column names must be lowercase

5. You want to process a large dataset using Snowpark and Python. Why is it better to write your transformation code in Snowpark rather than exporting data to your local machine?

hard

A. Snowpark runs code inside Snowflake, reducing data transfer and improving security.

B. Exporting data is faster and more secure than running code inside Snowflake.

C. Snowpark only supports SQL, so Python code must run locally.

D. Local processing avoids Snowflake compute costs.

Solution

1. Step 1: Understand Snowpark's architecture

   Snowpark runs code inside Snowflake close to the data storage.

2. Step 2: Benefits of running code inside Snowflake

   This reduces data transfer time and keeps data secure by not moving it outside.

3. Final Answer:

   Snowpark runs code inside Snowflake, reducing data transfer and improving security. -> Option A

4. Quick Check:

   Code near data = less transfer + better security [OK]

Hint: Run code inside Snowflake to keep data secure and fast [OK]

Common Mistakes:

• Thinking exporting data is faster or safer
• Believing Snowpark only supports SQL
• Ignoring Snowflake compute costs benefits