awk basics (field processing) in Linux CLI - Commands & Configuration

Introduction

Sometimes you have text files or command outputs with columns of data. Awk helps you pick out and work with these columns easily. It reads each line, splits it into fields, and lets you print or manipulate those fields.

When you want to extract the second column from a list of users and their IDs.

When you need to sum up numbers in the third column of a sales report.

When you want to print only the first and last names from a list separated by spaces.

When you want to filter lines based on a value in a specific column.

When you want to quickly reformat columns from a log file for easier reading.

Commands

This command prints the second word (field) from each line of the input text. Awk splits each line by spaces by default and $2 means the second field.

Terminal

echo -e "apple banana cherry\ndog elephant frog" | awk '{print $2}'

Expected Output

banana

elephant

This command adds the first and third numbers on each line and prints the result. Awk treats fields as numbers when used with arithmetic.

Terminal

echo -e "10 20 30\n40 50 60" | awk '{print $1 + $3}'

Expected Output

40

100

This prints the first and second fields (first and last names) separated by a space from each line.

Terminal

echo -e "John Doe 25\nJane Smith 30" | awk '{print $1, $2}'

Expected Output

John Doe

Jane Smith

This filters lines where the second field is greater than 6 and prints the first field from those lines.

Terminal

echo -e "red 5\nblue 10\ngreen 7" | awk '$2 > 6 {print $1}'

Expected Output

blue

green

Key Concept

If you remember nothing else from awk, remember: $1, $2, $3... represent the first, second, third fields of each line, and you can print or use them in calculations.

Common Mistakes

Using $0 when intending to print a specific field

$0 prints the whole line, not a single field, so you get the entire line instead of just one column.

Use $1, $2, etc., to print specific fields instead of $0.

Forgetting that awk splits fields by spaces or tabs by default

If your data uses a different separator like commas, awk will not split fields correctly and output will be wrong.

Use the -F flag to specify the correct field separator, e.g., awk -F, for comma-separated values.

Trying to do arithmetic on fields that contain text

Awk treats non-numeric fields as zero in arithmetic, leading to wrong results.

Make sure fields used in calculations contain only numbers.

Summary

Use awk with $1, $2, $3 to access fields (columns) in each line of text.

You can print fields, do math on numeric fields, or filter lines based on field values.

Awk splits fields by spaces or tabs by default but you can change this with -F.