COPY command for bulk data loading in PostgreSQL - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of the COPY command in PostgreSQL?

The COPY command is used to quickly load large amounts of data into a PostgreSQL table from a file or to export data from a table to a file.

beginner

How do you use the COPY command to load data from a CSV file into a table?

You use:
COPY table_name FROM '/path/to/file.csv' WITH (FORMAT csv, HEADER true);
This loads CSV data with a header row into the table.

beginner

What is the difference between COPY FROM and COPY TO?

COPY FROM loads data into a table from a file.
COPY TO exports data from a table to a file.

intermediate

Can the COPY command read from or write to a client program instead of a file?

Yes, using STDIN and STDOUT, COPY can read data from or write data to the client program, allowing interactive or programmatic data transfer.

intermediate

What permissions are required to use the COPY command with a file path?

The PostgreSQL server process must have read permission on the file for COPY FROM, and write permission for COPY TO. Also, the user must have appropriate table privileges.

Which command loads data from a CSV file into a PostgreSQL table?

A COPY table_name FROM '/path/file.csv' WITH (FORMAT csv, HEADER true);

B INSERT INTO table_name VALUES ('/path/file.csv');

C LOAD DATA INFILE '/path/file.csv' INTO table_name;

D SELECT * FROM '/path/file.csv';

COPY with FROM and FORMAT csv loads CSV data into the table.

What does the HEADER option do in the COPY command?

A Skips the first row of the file because it contains column names.

B Includes the first row as data.

C Specifies the file has no header.

D Encrypts the data during copy.

HEADER true tells COPY to skip the first row as it contains column headers.

Which of these is NOT a valid COPY command usage?

A COPY table_name TO '/path/file.csv' WITH (FORMAT csv);

B COPY table_name FROM STDIN;

C COPY table_name FROM '/path/file.csv' WITH (FORMAT json);

D COPY table_name FROM '/path/file.csv' WITH (FORMAT csv);

PostgreSQL COPY does not support JSON format directly.

What must the PostgreSQL server have to read a file with COPY FROM?

A Read permission on the file.

B Write permission on the file.

C Execute permission on the file.

D No permissions needed.

The server needs read permission to access the file for COPY FROM.

How can you export data from a table to a CSV file?

A COPY table_name TO '/path/file.csv' WITH (FORMAT csv, HEADER true);

B COPY table_name FROM '/path/file.csv';

C INSERT INTO table_name VALUES ('/path/file.csv');

D EXPORT table_name TO CSV;

COPY TO with FORMAT csv and HEADER true exports table data to a CSV file.

Explain how to use the COPY command to load data from a CSV file into a PostgreSQL table.

Think about the command structure and options for CSV files.

Describe the permissions and security considerations when using the COPY command with files.

Consider who needs access to the file and the database.

Practice

1. What is the main purpose of the COPY command in PostgreSQL?

easy

A. To delete data from a table

B. To create a new table in the database

C. To quickly load or export large amounts of data between a file and a table

D. To update existing rows in a table

Solution

1. Step 1: Understand the function of COPY

   The COPY command is designed to transfer data efficiently between a file and a PostgreSQL table.

2. Step 2: Compare with other commands

   Unlike CREATE, UPDATE, or DELETE, COPY focuses on bulk data loading or exporting, not modifying table structure or individual rows.

3. Final Answer:

   To quickly load or export large amounts of data between a file and a table -> Option C

4. Quick Check:

   COPY = bulk data transfer [OK]

Hint: COPY is for fast bulk data load/export, not table changes [OK]

Common Mistakes:

• Confusing COPY with CREATE TABLE
• Thinking COPY updates rows
• Assuming COPY deletes data

2. Which of the following is the correct syntax to copy data from a CSV file into a PostgreSQL table named employees?

easy

A. COPY employees TO '/path/to/file.csv' WITH CSV HEADER;

B. COPY employees FROM '/path/to/file.csv' WITH CSV HEADER;

C. COPY employees FROM '/path/to/file.csv' DELIMITER ';';

D. COPY employees FROM '/path/to/file.csv' FORMAT JSON;

Solution

1. Step 1: Identify direction of COPY

   To load data into a table, use COPY ... FROM. COPY employees FROM '/path/to/file.csv' WITH CSV HEADER; and C use FROM, B uses TO which exports data.

2. Step 2: Check format and options

   COPY employees FROM '/path/to/file.csv' WITH CSV HEADER; uses WITH CSV HEADER which is correct for CSV files with headers. COPY employees FROM '/path/to/file.csv' DELIMITER ';'; uses DELIMITER ';' but no CSV keyword, which is less clear. COPY employees FROM '/path/to/file.csv' FORMAT JSON; uses FORMAT JSON which is invalid for COPY.

3. Final Answer:

   COPY employees FROM '/path/to/file.csv' WITH CSV HEADER; -> Option B

4. Quick Check:

   Load CSV with header = COPY ... FROM ... WITH CSV HEADER [OK]

Hint: Use COPY ... FROM for loading, WITH CSV HEADER for CSV files [OK]

Common Mistakes:

• Using COPY ... TO instead of FROM to load data
• Missing WITH CSV HEADER for CSV files with headers
• Using wrong format like JSON in COPY

3. Given the table products(id INT, name TEXT, price NUMERIC), what will be the result of this command?

COPY products FROM '/tmp/products.csv' WITH CSV HEADER;

Assuming /tmp/products.csv contains:

id,name,price
1,Pen,1.20
2,Notebook,2.50
3,Eraser,0.80

medium

A. No rows inserted because HEADER skips all data

B. Error because price is numeric but CSV has strings

C. Only 2 rows inserted because of header misread

D. 3 rows inserted into products table

Solution

1. Step 1: Understand COPY with CSV HEADER

   The HEADER option tells COPY to skip the first line (column names) and load the rest as data rows.

2. Step 2: Check data types and rows

   The CSV data matches the table columns and types correctly: id as int, name as text, price as numeric. All 3 data rows will be inserted.

3. Final Answer:

   3 rows inserted into products table -> Option D

4. Quick Check:

   CSV HEADER skips first line, 3 data lines inserted [OK]

Hint: CSV HEADER skips first line; all data rows load if types match [OK]

Common Mistakes:

• Thinking HEADER skips all rows
• Assuming numeric columns reject string input
• Confusing COPY direction or syntax

4. You run this command:

COPY customers FROM '/data/customers.csv' WITH CSV;

But get an error: ERROR: could not open file "/data/customers.csv": Permission denied
What is the most likely cause?

medium

A. The PostgreSQL server process lacks permission to read the file

B. The CSV file format is incorrect

C. The COPY command syntax is wrong

D. The customers table does not exist

Solution

1. Step 1: Analyze the error message

   The error says "Permission denied" when opening the file, indicating a file access issue.

2. Step 2: Understand file permissions in COPY

   PostgreSQL server must have OS-level read permission on the file. If it lacks permission, this error occurs.

3. Final Answer:

   The PostgreSQL server process lacks permission to read the file -> Option A

4. Quick Check:

   Permission denied = file access issue [OK]

Hint: Permission denied means PostgreSQL can't read the file [OK]

Common Mistakes:

• Assuming syntax error causes permission denied
• Thinking CSV format causes file access error
• Ignoring file permissions on server

5. You want to load data from a CSV file into a table but skip rows where the age column is empty or NULL. Which approach using COPY and PostgreSQL features is best?

hard

A. Use COPY to load all data into a staging table, then insert filtered rows into the target table

B. Use COPY with WHERE clause to skip rows with empty age

C. Use COPY with an option to ignore NULL values in age column

D. Modify the CSV file manually to remove rows with empty age before COPY

Solution

1. Step 1: Understand COPY limitations

   COPY does not support filtering rows or WHERE clauses during import.

2. Step 2: Use staging table for filtering

   Load all data into a temporary or staging table, then run an INSERT INTO ... SELECT ... WHERE age IS NOT NULL to filter rows.

3. Step 3: Consider manual CSV editing

   While manual editing works, it is inefficient and error-prone for large files.

4. Final Answer:

   Use COPY to load all data into a staging table, then insert filtered rows into the target table -> Option A

5. Quick Check:

   COPY can't filter; use staging table + filter [OK]

Hint: COPY can't filter; load then filter with SQL [OK]

Common Mistakes:

• Expecting COPY to filter rows directly
• Looking for non-existent COPY options to ignore NULLs
• Relying on manual CSV edits for large data