COPY INTO command in Snowflake - Mini Project: Build & Apply

Loading Data into Snowflake Using COPY INTO Command

📖 Scenario: You are working as a data engineer. You have a CSV file stored in a Snowflake stage. Your task is to load this data into a Snowflake table using the COPY INTO command.

🎯 Goal: Build a Snowflake script that creates a table, sets up a stage, and uses the COPY INTO command to load data from a CSV file into the table.

📋 What You'll Learn

Create a table named employees with columns id (integer), name (string), and salary (number).

Create a named stage called my_csv_stage pointing to an internal location.

Use the COPY INTO command to load data from a CSV file named employees.csv in the stage into the employees table.

Specify the file format as CSV with header row skipped.

💡 Why This Matters

🌍 Real World

Loading data from files into Snowflake tables is a common task in data engineering pipelines to prepare data for analysis.

💼 Career

Understanding how to use the COPY INTO command and manage stages and file formats is essential for roles like data engineer, cloud data architect, and database administrator working with Snowflake.

1

Create the employees table

Write a SQL statement to create a table called employees with columns id as INTEGER, name as VARCHAR, and salary as NUMBER.

Snowflake

-- Create the employees table
-- Your code here

Hint

Use CREATE OR REPLACE TABLE employees and define the three columns with their types.

2

Create an internal stage named my_csv_stage

Write a SQL statement to create or replace an internal stage called my_csv_stage.

Snowflake

CREATE OR REPLACE TABLE employees (
    id INTEGER,
    name VARCHAR,
    salary NUMBER
);

-- Create the internal stage my_csv_stage
-- Your code here

Hint

Use CREATE OR REPLACE STAGE my_csv_stage; to create the stage.

3

Write the COPY INTO command to load data

Write a COPY INTO command to load data from the file employees.csv in the stage my_csv_stage into the employees table. Specify the file format as CSV and skip the header row.

Snowflake

CREATE OR REPLACE TABLE employees (
    id INTEGER,
    name VARCHAR,
    salary NUMBER
);

CREATE OR REPLACE STAGE my_csv_stage;

-- Write COPY INTO command to load employees.csv from my_csv_stage
-- Your code here

Hint

Use COPY INTO employees FROM @my_csv_stage/employees.csv and specify the file format with SKIP_HEADER = 1.

4

Add a file format object for CSV with header skip

Create or replace a file format named csv_format with type CSV and skip the header row. Then update the COPY INTO command to use this file format by name.

Snowflake

CREATE OR REPLACE TABLE employees (
    id INTEGER,
    name VARCHAR,
    salary NUMBER
);

CREATE OR REPLACE STAGE my_csv_stage;

-- Create file format csv_format and update COPY INTO to use it
-- Your code here

Hint

First create the file format with CREATE OR REPLACE FILE FORMAT csv_format TYPE = 'CSV' SKIP_HEADER = 1;. Then use FILE_FORMAT = (FORMAT_NAME = 'csv_format') in the COPY INTO command.

Practice

1. What is the main purpose of the COPY INTO command in Snowflake?

easy

A. To load data files from cloud storage into Snowflake tables

B. To export data from Snowflake tables to local files

C. To create new tables in Snowflake

D. To delete data from Snowflake tables

Solution

1. Step 1: Understand the command purpose

   The COPY INTO command is designed to load data from external files into Snowflake tables.

2. Step 2: Compare options

   Options B, C, and D describe exporting, creating, or deleting data, which are not the function of COPY INTO.

3. Final Answer:

   To load data files from cloud storage into Snowflake tables -> Option A

4. Quick Check:

   Loading data = COPY INTO [OK]

Hint: COPY INTO means loading data into tables [OK]

Common Mistakes:

• Confusing COPY INTO with export commands
• Thinking COPY INTO creates tables
• Assuming COPY INTO deletes data

2. Which of the following is the correct basic syntax to load data using COPY INTO in Snowflake?

easy

A. COPY table_name INTO @stage_name FILE_FORMAT = (TYPE = 'CSV');

B. LOAD DATA INTO table_name FROM @stage_name FORMAT = CSV;

C. COPY INTO table_name FROM @stage_name FILE_FORMAT = (TYPE = 'CSV');

D. INSERT INTO table_name COPY FROM @stage_name FILE_FORMAT = CSV;

Solution

1. Step 1: Recall correct COPY INTO syntax

   The correct syntax is COPY INTO table_name FROM @stage_name FILE_FORMAT = (TYPE = 'CSV'); to load CSV files from a stage.

2. Step 2: Identify incorrect syntax

   Options A, C, and D use wrong keywords or order that Snowflake does not accept.

3. Final Answer:

   COPY INTO table_name FROM @stage_name FILE_FORMAT = (TYPE = 'CSV'); -> Option C

4. Quick Check:

   Correct COPY INTO syntax = COPY INTO table_name FROM @stage_name FILE_FORMAT = (TYPE = 'CSV'); [OK]

Hint: COPY INTO table FROM stage with FILE_FORMAT [OK]

Common Mistakes:

• Swapping INTO and FROM keywords
• Using LOAD DATA instead of COPY INTO
• Incorrect FILE_FORMAT syntax

3. Given the command:

COPY INTO my_table FROM @my_stage FILE_FORMAT = (TYPE = 'CSV' FIELD_DELIMITER = ',') ON_ERROR = 'skip_file';

What happens if one file in the stage has errors during loading?

medium

A. Snowflake retries loading the file indefinitely

B. The entire load fails and no data is loaded

C. Only the erroneous rows are skipped, file loads partially

D. The file with errors is skipped, other files load successfully

Solution

1. Step 1: Understand ON_ERROR = 'skip_file'

   This option tells Snowflake to skip any file that causes errors during loading instead of failing the entire load.

2. Step 2: Analyze the effect on other files

   Other files without errors will load normally, so only the bad file is skipped.

3. Final Answer:

   The file with errors is skipped, other files load successfully -> Option D

4. Quick Check:

   ON_ERROR = skip_file skips bad files [OK]

Hint: ON_ERROR = skip_file skips bad files, loads others [OK]

Common Mistakes:

• Assuming entire load fails on one bad file
• Thinking only bad rows are skipped with skip_file
• Believing Snowflake retries endlessly

4. You run this command:

COPY INTO sales FROM @data_stage FILE_FORMAT = (FORMAT_NAME = 'csv_format');

But get an error: SQL compilation error: file format not found. What is the likely cause?

medium

A. The stage @data_stage does not exist

B. The FILE_FORMAT clause references a named file format that does not exist

C. The sales table does not exist

D. The CSV files are empty

Solution

1. Step 1: Analyze the error message

   The error says file format not found, meaning Snowflake expects a named file format or a full definition.

2. Step 2: Check the FILE_FORMAT clause

   The clause FILE_FORMAT = (FORMAT_NAME = 'csv_format') references a named file format that does not exist.

3. Final Answer:

   The FILE_FORMAT clause references a named file format that does not exist -> Option B

4. Quick Check:

   File format error means missing or wrong FILE_FORMAT [OK]

Hint: File format errors mean missing or wrong FILE_FORMAT [OK]

Common Mistakes:

• Assuming stage or table missing causes file format error
• Ignoring need for file format definition
• Thinking empty files cause this error

5. You want to load JSON files from an external S3 bucket into a Snowflake table named events. Which COPY INTO command correctly handles JSON format and ignores files with errors?

hard

A. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'JSON') ON_ERROR = 'skip_file';

B. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'CSV') ON_ERROR = 'continue';

C. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'JSON') ON_ERROR = 'abort_statement';

D. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'XML') ON_ERROR = 'skip_file';

Solution

1. Step 1: Match file format to JSON files

   Since files are JSON, the FILE_FORMAT must specify TYPE = 'JSON'.

2. Step 2: Choose error handling to ignore bad files

   ON_ERROR = 'skip_file' skips files with errors, which matches the requirement.

3. Step 3: Eliminate incorrect options

   COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'CSV') ON_ERROR = 'continue'; uses CSV format, wrong for JSON. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'JSON') ON_ERROR = 'abort_statement'; aborts on error, not ignoring. COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'XML') ON_ERROR = 'skip_file'; uses XML format, incorrect.

4. Final Answer:

   COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'JSON') ON_ERROR = 'skip_file'; -> Option A

5. Quick Check:

   JSON format + skip_file error handling = COPY INTO events FROM @s3_stage FILE_FORMAT = (TYPE = 'JSON') ON_ERROR = 'skip_file'; [OK]

Hint: Match FILE_FORMAT type to file type and use skip_file to ignore errors [OK]

Common Mistakes:

• Using wrong file format type
• Choosing abort instead of skip_file for errors
• Confusing JSON with CSV or XML