awk patterns and actions in Linux CLI - Commands & Configuration

Introduction

Sometimes you want to find and process specific parts of text in files or command output. Awk helps by letting you match patterns and then do actions on those matches, like printing or calculating.

When you want to print only lines containing a certain word from a log file.

When you need to sum numbers in a specific column of a data file.

When you want to extract and reformat parts of text from command output.

When you want to count how many lines match a pattern in a file.

When you want to replace or modify text only on lines that match a pattern.

Commands

This command searches the syslog file for lines containing the word 'error' and prints those entire lines. The pattern is '/error/' and the action is '{print $0}', which means print the whole line.

Terminal

awk '/error/ {print $0}' /var/log/syslog

Expected Output

Apr 26 10:15:32 server1 kernel: [12345.678901] error: device not found

Apr 26 10:16:45 server1 systemd: error: failed to start service

This command reads 'data.txt' and prints the first and third columns only for lines where the third column is greater than 50. The pattern is '$3 > 50' and the action is '{print $1, $3}'.

Terminal

awk '$3 > 50 {print $1, $3}' data.txt

Expected Output

John 75

Alice 82

This command sums all numbers in the second column of 'numbers.txt'. The BEGIN block initializes sum to 0, the main block adds each second column value to sum, and the END block prints the total sum.

Terminal

awk 'BEGIN {sum=0} {sum += $2} END {print sum}' numbers.txt

Expected Output

210

This command pipes three fruit names to awk, which prints only lines containing the letter 'a'. It shows how awk can process piped input with a pattern and action.

Terminal

echo -e "apple\nbanana\ncherry" | awk '/a/ {print $0}'

Expected Output

apple

banana

Key Concept

If you remember nothing else from awk patterns and actions, remember: awk looks at each line, checks if it matches the pattern, and if yes, runs the action on that line.

Common Mistakes

Using single quotes inside the awk pattern without escaping

This breaks the shell command because the shell misinterprets the quotes.

Use double quotes for the shell command or escape single quotes inside the awk script.

Forgetting to specify an action, expecting awk to print matching lines automatically

Awk prints the whole line only if no action is given; if you provide an empty action block, nothing prints.

Either omit the action block to print matching lines or explicitly use '{print $0}'.

Using incorrect field numbers like $0 for a specific column

$0 means the whole line, not a column; using it for a column causes wrong output.

Use $1, $2, $3, etc., for columns; use $0 only to refer to the entire line.

Summary

Use awk patterns to select lines based on text or conditions.

Use actions to print, calculate, or modify data on matching lines.

Combine BEGIN and END blocks to initialize and finalize processing.

Practice

1. In awk, what does a pattern do?

easy

A. It selects which lines to process

B. It changes the file content permanently

C. It saves the output to a file

D. It runs a shell command

Solution

1. Step 1: Understand the role of patterns in awk

   Patterns in awk are used to match lines in the input data. Only lines matching the pattern are processed.

2. Step 2: Differentiate patterns from actions

   Actions define what to do with matched lines, but patterns decide which lines to select.

3. Final Answer:

   It selects which lines to process -> Option A

4. Quick Check:

   Pattern = line selector [OK]

Hint: Patterns filter lines before actions run [OK]

Common Mistakes:

• Thinking patterns change file content
• Confusing patterns with output commands
• Assuming patterns run shell commands

2. Which of these is the correct syntax to print the first field of each line using awk?

easy

A. awk '{print $}' file.txt

B. awk 'print $1' file.txt

C. awk '{print 1}' file.txt

D. awk '{print $1}' file.txt

Solution

1. Step 1: Identify correct awk print syntax

   The print statement must be inside braces {} and use $1 to refer to the first field.

2. Step 2: Check each option

   awk '{print $1}' file.txt uses correct braces and $1. awk 'print $1' file.txt misses braces. awk '{print 1}' file.txt prints number 1 literally. awk '{print $}' file.txt uses invalid $.

3. Final Answer:

   awk '{print $1}' file.txt -> Option D

4. Quick Check:

   Print first field = awk '{print $1}' [OK]

Hint: Always use braces {} around actions in awk [OK]

Common Mistakes:

• Omitting braces around print
• Using $ without field number
• Printing literal numbers instead of fields

3. What is the output of this command on a file with lines:
1: apple 10
2: banana 20
3: cherry 30

awk '$2 > 15 {print $1}' file.txt?

medium

A. apple

B. banana
cherry

C. 10
20
30

D. apple
banana
cherry

Solution

1. Step 1: Understand the pattern condition

   The pattern '$2 > 15' means only lines where the second field is greater than 15 are selected.

2. Step 2: Apply the condition to each line

   Line 1: 10 > 15? No. Line 2: 20 > 15? Yes. Line 3: 30 > 15? Yes.

3. Step 3: Print the first field of matching lines

   For lines 2 and 3, print $1: 'banana' and 'cherry'.

4. Final Answer:

   banana cherry -> Option B

5. Quick Check:

   Fields > 15 print first field = banana, cherry [OK]

Hint: Check pattern filters before printing fields [OK]

Common Mistakes:

• Printing all lines ignoring pattern
• Printing second field instead of first
• Confusing numeric comparison with string

4. Identify the error in this awk command:
awk '$1 == "apple" print $2' file.txt

medium

A. Incorrect field number

B. Wrong comparison operator

C. Missing braces {} around action

D. Quotes around apple should be single quotes

Solution

1. Step 1: Check awk syntax for pattern and action

   Awk requires actions to be inside braces {} after the pattern.

2. Step 2: Identify missing braces

   The command misses braces around 'print $2', causing syntax error.

3. Final Answer:

   Missing braces {} around action -> Option C

4. Quick Check:

   Actions need braces {} [OK]

Hint: Always put actions inside {} in awk [OK]

Common Mistakes:

• Omitting braces around actions
• Using wrong comparison operators
• Misquoting strings inside patterns

5. You want to print the second field only for lines where the first field starts with 'a' or 'b'. Which awk command achieves this?

hard

A. awk '$1 ~ /^[ab]/ {print $2}' file.txt

B. awk '$1 == "a" || $1 == "b" {print $2}' file.txt

C. awk '$1 = "a" || $1 = "b" {print $2}' file.txt

D. awk '$1 ~ /a|b/ {print $2}' file.txt

Solution

1. Step 1: Understand pattern for fields starting with 'a' or 'b'

   The regex /^[ab]/ matches strings starting with 'a' or 'b'. The operator ~ means regex match.

2. Step 2: Check each option's correctness

   awk '$1 ~ /^[ab]/ {print $2}' file.txt uses correct regex and syntax. awk '$1 == "a" || $1 == "b" {print $2}' file.txt compares whole field to 'a' or 'b' exactly, not starts with. awk '$1 = "a" || $1 = "b" {print $2}' file.txt uses assignment '=' instead of comparison '=='. awk '$1 ~ /a|b/ {print $2}' file.txt matches anywhere 'a' or 'b' appears, not just start.

3. Final Answer:

   awk '$1 ~ /^[ab]/ {print $2}' file.txt -> Option A

4. Quick Check:

   Regex match start with a or b = awk '$1 ~ /^[ab]/ {print $2}' file.txt [OK]

Hint: Use regex ^[ab] to match start of field [OK]

Common Mistakes:

• Using = instead of == for comparison
• Matching exact string instead of start
• Using wrong regex without ^ anchor