Why Snowpark brings code to the data in Snowflake - See It in Action

Why Snowpark Brings Code to the Data

📖 Scenario: You work at a company that stores large amounts of data in Snowflake. You want to process this data efficiently without moving it around.

🎯 Goal: Build a simple Snowpark program that shows how to bring code to the data inside Snowflake, instead of moving data out for processing.

📋 What You'll Learn

Create a Snowpark session connected to Snowflake

Create a DataFrame from an existing table called sales_data

Define a simple transformation function using Snowpark

Apply the function to the DataFrame and collect results

💡 Why This Matters

🌍 Real World

Companies use Snowpark to run data processing close to where data lives in Snowflake. This reduces data movement, speeds up processing, and improves security.

💼 Career

Understanding Snowpark is valuable for data engineers and cloud architects who build scalable data pipelines and analytics solutions on Snowflake.

1

Create a Snowpark session

Write code to create a Snowpark session called session using the Session.builder.configs() method with a dictionary connection_parameters containing your Snowflake account details.

Snowflake

# Your code here to create a Snowpark session named session

Hint

Use Session.builder.configs(connection_parameters).create() to create the session.

2

Create a DataFrame from the sales_data table

Use the session.table() method to create a DataFrame called df from the existing table named sales_data.

Snowflake

session = Session.builder.configs(connection_parameters).create()
# Your code here to create a DataFrame named df from sales_data

Hint

Use df = session.table("sales_data") to create the DataFrame.

3

Define a transformation function using Snowpark

Define a function called add_discount that takes a DataFrame df and returns a new DataFrame with a new column discounted_price calculated as price * 0.9.

Snowflake

df = session.table("sales_data")
# Your code here to define add_discount(df) function

Hint

Use with_column to add the new column inside the function.

4

Apply the function and collect results

Call the add_discount function with df and assign the result to df_discounted. Then use df_discounted.collect() to fetch the results into a variable called results.

Snowflake

def add_discount(df):
    return df.with_column("discounted_price", df["price"] * 0.9)
# Your code here to apply add_discount and collect results

Hint

Assign the function result to df_discounted and then call collect() to get results.

Practice

1. What is the main benefit of Snowpark bringing code to the data inside Snowflake?

easy

A. It increases data duplication across systems.

B. It requires exporting data to external servers.

C. It only supports SQL queries.

D. It reduces data movement and improves performance.

Solution

1. Step 1: Understand Snowpark's design

   Snowpark runs code inside Snowflake, close to the data storage.

2. Step 2: Identify benefits of running code near data

   This reduces the need to move large data sets, saving time and resources.

3. Final Answer:

   It reduces data movement and improves performance. -> Option D

4. Quick Check:

   Code near data = less data movement [OK]

Hint: Code runs inside Snowflake, so data stays put [OK]

Common Mistakes:

• Thinking data must be exported to run code
• Believing Snowpark only supports SQL
• Assuming data is duplicated across systems

2. Which of the following is the correct way to create a Snowpark session in Python?

easy

A. session = SnowflakeSession.create(connection_parameters)

B. session = snowpark.connect(connection_parameters)

C. session = snowflake.snowpark.Session.builder.configs(connection_parameters).create()

D. session = snowflake.connect_snowpark(connection_parameters)

Solution

1. Step 1: Recall Snowpark Python session creation syntax

   Snowpark uses Session.builder.configs(...).create() to start a session.

2. Step 2: Match the correct method call

   session = snowflake.snowpark.Session.builder.configs(connection_parameters).create() matches the official syntax for creating a session in Snowpark Python API.

3. Final Answer:

   session = snowflake.snowpark.Session.builder.configs(connection_parameters).create() -> Option C

4. Quick Check:

   Session.builder.configs(...).create() = correct syntax [OK]

Hint: Look for builder.configs(...).create() pattern [OK]

Common Mistakes:

• Using incorrect method names like connect or connect_snowpark
• Missing builder or configs in the call
• Confusing Snowpark session with generic Snowflake connection

3. Given this Snowpark Python code snippet:

df = session.table('EMPLOYEES')
filtered = df.filter(df['DEPARTMENT'] == 'SALES')
count = filtered.count()

What does count represent?

medium

A. The number of rows where DEPARTMENT is 'SALES'.

B. The number of columns in the EMPLOYEES table.

C. The total number of rows in the EMPLOYEES table.

D. The number of distinct departments in EMPLOYEES.

Solution

1. Step 1: Understand the filter operation

   The code filters rows where DEPARTMENT equals 'SALES'.

2. Step 2: Understand the count method

   Count returns the number of rows in the filtered DataFrame.

3. Final Answer:

   The number of rows where DEPARTMENT is 'SALES'. -> Option A

4. Quick Check:

   filter + count = rows matching condition [OK]

Hint: Filter then count means count matching rows [OK]

Common Mistakes:

• Thinking count returns total rows without filter
• Confusing count with number of columns
• Assuming count returns distinct values

4. You wrote this Snowpark code but get an error:

df = session.table('EMPLOYEES')
filtered = df.filter(df.DEPARTMENT = 'SALES')
count = filtered.count()

What is the error and how to fix it?

medium

A. Add parentheses around the filter condition.

B. Use '==' for comparison instead of '=' in the filter condition.

C. Rename 'DEPARTMENT' column to lowercase.

D. Change 'filter' to 'where' method.

Solution

1. Step 1: Identify the filter condition syntax

   The filter uses '=' which is assignment, not comparison.

2. Step 2: Correct the comparison operator

   Use '==' to compare column values in Snowpark expressions.

3. Final Answer:

   Use '==' for comparison instead of '=' in the filter condition. -> Option B

4. Quick Check:

   Comparison needs '==' not '=' [OK]

Hint: Use '==' for equality checks in filters [OK]

Common Mistakes:

• Using single '=' instead of '==' in conditions
• Thinking 'where' is required instead of 'filter'
• Assuming column names must be lowercase

5. You want to process a large dataset using Snowpark and Python. Why is it better to write your transformation code in Snowpark rather than exporting data to your local machine?

hard

A. Snowpark runs code inside Snowflake, reducing data transfer and improving security.

B. Exporting data is faster and more secure than running code inside Snowflake.

C. Snowpark only supports SQL, so Python code must run locally.

D. Local processing avoids Snowflake compute costs.

Solution

1. Step 1: Understand Snowpark's architecture

   Snowpark runs code inside Snowflake close to the data storage.

2. Step 2: Benefits of running code inside Snowflake

   This reduces data transfer time and keeps data secure by not moving it outside.

3. Final Answer:

   Snowpark runs code inside Snowflake, reducing data transfer and improving security. -> Option A

4. Quick Check:

   Code near data = less transfer + better security [OK]

Hint: Run code inside Snowflake to keep data secure and fast [OK]

Common Mistakes:

• Thinking exporting data is faster or safer
• Believing Snowpark only supports SQL
• Ignoring Snowflake compute costs benefits