Processing CSV files in Bash Scripting - Practice Problems & Coding Challenges

Challenge - 5 Problems

💻 Command Output

intermediate

Extract the second column from a CSV file

Given a CSV file data.csv with the following content:

name,age,city
Alice,30,New York
Bob,25,Los Angeles
Carol,27,Chicago

Which command outputs only the age column values (excluding the header)?

A tail -n +2 data.csv | cut -d',' -f2

B cut -d',' -f2 data.csv

C awk -F',' '{print $2}' data.csv

D sed '1d' data.csv | cut -d',' -f3

💡 Hint

Use a command to skip the header line, then extract the second column.

Option A skips the first line (header) using tail -n +2 and then extracts the second column with cut -d',' -f2. Option A includes the header. Option A prints the header too. Option A extracts the third column, not the second.

💻 Command Output

intermediate

Count rows where the city is 'Chicago'

Using the same data.csv file, which command outputs the number of rows where the city is exactly Chicago?

A grep -c '^.*,.*,Chicago$' data.csv

B sed -n '/Chicago/p' data.csv | wc -l

C awk -F',' '$3 == "Chicago" {count++} END {print count}' data.csv

D grep -c 'Chicago' data.csv

💡 Hint

Use a tool that can compare exact field values.

Option C uses awk to check if the third field equals 'Chicago' exactly and counts matches. Option C matches lines ending with Chicago but includes header if it matches. Option C counts any line with 'Chicago' anywhere, including header. Option C counts lines containing 'Chicago' but includes header if present.

🔧 Debug

advanced

Fix the script to sum ages from CSV

This script aims to sum all ages from data.csv (excluding header):

total=0
while IFS=',' read name age city; do
  total=$((total + age))
done < data.csv
echo $total

What is the main reason this script does not produce the correct sum?

Bash Scripting

total=0
while IFS=',' read name age city; do
  total=$((total + age))
done < data.csv
echo $total

A The script includes the header line, causing a non-numeric addition error.

B The IFS variable is not set correctly to split CSV fields.

C The variable 'total' is not initialized before the loop.

D The read command does not split fields because of missing quotes.

💡 Hint

Check if the header line is processed as data.

The script reads the header line 'name,age,city' and tries to add 'age' (a string) to total, causing an error or wrong result. Skipping the header fixes this.

🚀 Application

advanced

Extract unique cities sorted alphabetically

Which command pipeline extracts all unique city names from data.csv (excluding header) and sorts them alphabetically?

A sed '1d' data.csv | cut -d',' -f3 | sort -r | uniq

B cut -d',' -f3 data.csv | sort -u

C tail -n +2 data.csv | cut -d',' -f3 | sort | uniq

D awk -F',' 'NR>1 {print $3}' data.csv | sort | uniq

💡 Hint

Skip the header, extract the third column, then sort and remove duplicates.

Option D uses awk to skip the header (NR>1), prints the third field, sorts, and removes duplicates with uniq. Option D works but uniq after sort is less efficient than sort -u. Option D includes header. Option D sorts in reverse order, not alphabetical.

🧠 Conceptual

expert

Why is parsing CSV with simple tools risky?

Why can using basic tools like cut or awk with a comma delimiter be unreliable for processing CSV files?

A Because CSV files always use tabs, not commas, as delimiters.

B Because CSV fields can contain commas inside quoted strings, breaking simple delimiter splitting.

C Because these tools do not support reading files larger than 1MB.

D Because these tools cannot handle files with more than 3 columns.

💡 Hint

Think about how commas inside quotes affect splitting.

CSV fields may contain commas inside quotes, so splitting by commas blindly breaks fields. Proper CSV parsers handle quoted fields correctly. Options A, B, and D are incorrect statements.

Practice

1. What does the cut -d, -f2 file.csv command do when processing a CSV file?

easy

A. Deletes the second column from the CSV file

B. Extracts the second column from the CSV file using comma as delimiter

C. Counts the number of commas in the CSV file

D. Replaces commas with tabs in the CSV file

Solution

1. Step 1: Understand the cut command with delimiter

   The -d, option sets the delimiter to a comma, which is the separator in CSV files.

2. Step 2: Extract the specified field

   The -f2 option extracts the second field (column) from each line of the file.

3. Final Answer:

   Extracts the second column from the CSV file using comma as delimiter -> Option B

4. Quick Check:

   cut -d, -f2 extracts column 2 [OK]

Hint: Use -d for delimiter and -f for field number in cut [OK]

Common Mistakes:

• Confusing -f with line number
• Using wrong delimiter character
• Thinking cut deletes columns

2. Which of the following is the correct syntax to print the third column of a CSV file using awk?

easy

A. awk -F, '{print $3}' file.csv

B. awk -d',' '{print $3}' file.csv

C. awk -F',' 'print $3' file.csv

D. awk '{print $3}' -F',' file.csv

Solution

1. Step 1: Identify the field separator option in awk

   The correct option to set the field separator is -F,, which tells awk to split fields by commas.

2. Step 2: Use the print statement correctly

   The print statement must be inside single quotes and include curly braces: {print $3} to print the third column.

3. Final Answer:

   awk -F, '{print $3}' file.csv -> Option A

4. Quick Check:

   Use -F for delimiter and print $3 inside braces [OK]

Hint: Use -F, and print $column inside braces in awk [OK]

Common Mistakes:

• Using -d instead of -F for delimiter
• Omitting curly braces around print
• Placing -F option after the script

3. Given a CSV file data.csv with content:

name,age,city
Alice,30,New York
Bob,25,Los Angeles
Carol,27,Chicago

What is the output of the command awk -F, '$2 > 26 {print $1}' data.csv?

medium

A. Alice

B. Alice
Bob
Carol

C. Bob
Carol

D. Alice
Carol

Solution

1. Step 1: Understand the awk condition and field separator

   The command uses -F, to split fields by commas. The condition $2 > 26 filters rows where the second field (age) is greater than 26.

2. Step 2: Apply the condition to each row

   Rows:
   - Header: 'age' > 26? No (string)
   - Alice,30: 30 > 26? Yes, print $1 = Alice
   - Bob,25: 25 > 26? No
   - Carol,27: 27 > 26? Yes, print $1 = Carol

3. Final Answer:

   Alice Carol -> Option D

4. Quick Check:

   Filter ages > 26, print names [OK]

Hint: Filter with $column condition and print desired field [OK]

Common Mistakes:

• Including header row in output
• Confusing field numbers
• Not using correct field separator

4. You wrote this command to extract the first column from a CSV file:
cut -f1 -d',' file.csv
But it returns the whole line instead of just the first column. What is the likely problem?

medium

A. The -f1 option is invalid

B. The file is empty

C. The delimiter option -d',' is missing or incorrect

D. cut cannot process CSV files

Solution

1. Step 1: Check the delimiter option usage

   The command uses -d',', which is incorrect. The single quotes make the delimiter the string ',' (quote comma quote) instead of plain comma (,). cut searches for ',' which isn't in the CSV.

2. Step 2: Understand cut behavior without correct delimiter

   Without splitting on ,, cut falls back to default (TAB), treating each line as one field, so -f1 outputs the entire line.

3. Final Answer:

   The delimiter option -d',' is missing or incorrect -> Option C

4. Quick Check:

   cut needs -d, (no quotes around comma) to split fields [OK]

Hint: Always specify -d, for CSV files with cut [OK]

Common Mistakes:

• Forgetting -d option
• Using wrong delimiter character
• Assuming cut auto-detects delimiter

5. You want to create a new CSV file containing only rows where the city is 'Chicago' from data.csv. Which bash command correctly filters and saves these rows?

hard

A. awk -F, '$3 == "Chicago"' data.csv > chicago.csv

B. grep 'Chicago' data.csv > chicago.csv

C. cut -d',' -f3 data.csv | grep 'Chicago' > chicago.csv

D. awk -F, '{if ($3 = "Chicago") print}' data.csv > chicago.csv

Solution

1. Step 1: Use awk with correct field and condition

   Using -F, sets comma as delimiter. The condition $3 == "Chicago" selects rows where the third column equals 'Chicago'.

2. Step 2: Redirect output to new file

   The output is redirected to chicago.csv using >, saving only matching rows.

3. Final Answer:

   awk -F, '$3 == "Chicago"' data.csv > chicago.csv -> Option A

4. Quick Check:

   Use awk with == for exact match and redirect output [OK]

Hint: Use awk with == for exact match and redirect output [OK]

Common Mistakes:

• Using single = instead of == in awk condition
• Using grep which may match partial fields
• Using cut which extracts columns, not rows