Processing CSV files in Bash Scripting

Introduction

CSV files store data in a simple table format. Processing them helps you extract or change information easily using scripts.

You want to count how many rows are in a CSV file.

You need to extract a specific column from a CSV file.

You want to filter rows based on a value in a column.

You want to convert CSV data into another format.

You want to automate reports from CSV data.

Syntax

Bash Scripting

cut -d',' -f<column_number> <file.csv>
awk -F',' '{print $<column_number>}' <file.csv>
grep '<pattern>' <file.csv>

-d',' tells commands the separator is a comma.

-f or $ followed by a number picks a column.

Examples

This extracts the second column from data.csv.

Bash Scripting

cut -d',' -f2 data.csv

This prints the third column from data.csv using awk.

Bash Scripting

awk -F',' '{print $3}' data.csv

This finds all lines containing the word 'John' in data.csv.

Bash Scripting

grep 'John' data.csv

Sample Program

This script reads data.csv, skips the header line, and prints the names (second column).

Bash Scripting

#!/bin/bash
# Script to print names from a CSV file
# CSV format: id,name,age

FILE=data.csv

if [ ! -f "$FILE" ]; then
  echo "File $FILE not found."
  exit 1
fi

# Print header
echo "Names in the file:"

# Skip header and print second column (name)
tail -n +2 "$FILE" | cut -d',' -f2

Important Notes

CSV files can have commas inside quotes; simple tools like cut may not handle those correctly.

For complex CSV parsing, consider tools like csvkit or scripting languages like Python.

Summary

CSV files store data separated by commas.

Use cut, awk, and grep to process CSV files in bash.

Simple commands work well for basic CSV files without quoted commas.