awk basics (field processing) in Linux CLI - Step-by-Step Execution

Concept Flow - awk basics (field processing)

Start: Read line from input

↓

Split line into fields

↓

Process fields using $1, $2, ...

↓

Perform action (print, calculate, etc.)

↓

Read next line or end if no more lines

Awk reads each line, splits it into fields, processes fields by position, then performs actions like printing.

Execution Sample

Linux CLI

echo "apple banana cherry" | awk '{print $2}'

Prints the second word (field) from the input line.

Execution Table

Step	Input Line	Fields Split	Field $1	Field $2	Field $3	Action	Output
1	apple banana cherry	[apple, banana, cherry]	apple	banana	cherry	print $2	banana
2	No more lines	N/A	N/A	N/A	N/A	End

💡 No more input lines to process, awk stops.

Variable Tracker

Variable	Start	After 1	Final
$1	N/A	apple	apple
$2	N/A	banana	banana
$3	N/A	cherry	cherry

Key Moments - 2 Insights

Why does awk use $1, $2, $3 instead of variable names?

Awk automatically splits each input line into fields numbered $1, $2, $3, etc. This lets you access words by position without naming them. See execution_table row 1 where fields are split and assigned.

What happens if the input line has fewer fields than the number used?

If you try to access a field number that doesn't exist, awk returns an empty string. For example, if $3 doesn't exist, printing $3 outputs nothing. This is shown by the variable_tracker where fields are only set if present.

Visual Quiz - 3 Questions

Look at the execution_table at step 1, what is the value of $3?

A apple

B cherry

C banana

D No value

💡 Hint

Check the 'Field $3' column in execution_table row 1.

At which step does awk stop processing input?

A Step 2

B Step 1

C After Step 1

D Never stops

💡 Hint

Look at the 'Action' column in execution_table row 2.

If the input line was "dog cat", what would $3 be?

A dog

B cat

C empty string

D error

💡 Hint

Refer to key_moments about missing fields returning empty strings.

Concept Snapshot

Awk reads input line by line.
Splits each line into fields by spaces.
Fields accessed by $1, $2, $3...
Use print $n to output fields.
If field missing, $n is empty string.
Processes until no more lines.

Full Transcript

Awk is a tool that reads text line by line. For each line, it splits the line into parts called fields, separated by spaces by default. Each field can be accessed by a number starting at 1, like $1 for the first word, $2 for the second, and so on. You can tell awk to do things with these fields, like print one of them. For example, printing $2 will show the second word of each line. If a line has fewer words than the number you ask for, awk gives an empty string for that field. Awk keeps doing this for every line until there are no more lines to read, then it stops.

Practice

1. What does $2 represent in an awk command?

easy

A. The number of fields in the current line

B. The entire current line

C. The first character of the line

D. The second field (word) in the current line

Solution

1. Step 1: Understand how awk splits lines

   awk automatically splits each input line into fields separated by spaces or tabs.

2. Step 2: Identify what $2 means

   $2 refers to the second field (word) in the current line.

3. Final Answer:

   The second field (word) in the current line -> Option D

4. Quick Check:

   $2 = second field [OK]

Hint: Remember: $1 is first word, $2 second, and so on [OK]

Common Mistakes:

• Confusing $2 with the whole line ($0)
• Thinking $2 is the number of fields (NF)
• Mixing field numbers with character positions

2. Which of the following is the correct syntax to print the third field of each line using awk?

easy

A. awk '{print $3}' filename

B. awk 'print $3' filename

C. awk '{print 3}' filename

D. awk '{print $}' filename

Solution

1. Step 1: Recall the basic awk print syntax

   The print statement must be inside braces: {print $3}.

2. Step 2: Check each option

   awk '{print $3}' filename uses correct braces and $3. Options B, C, and D have syntax errors or wrong usage.

3. Final Answer:

   awk '{print $3}' filename -> Option A

4. Quick Check:

   Print field 3 with braces [OK]

Hint: Always use braces { } around print commands in awk [OK]

Common Mistakes:

• Omitting braces around print statement
• Printing a number instead of a field variable
• Using incomplete or invalid field syntax

3. Given a file data.txt with the following content:

apple 10 red
banana 5 yellow
cherry 20 red

What is the output of awk '{print $1, $3}' data.txt?

medium

A. apple red
banana yellow
cherry red

B. apple
banana
cherry

C. 10 red
5 yellow
20 red

D. apple 10
banana 5
cherry 20

Solution

1. Step 1: Identify fields in each line

   Each line has three fields: $1=fruit, $2=number, $3=color.

2. Step 2: Understand the print command

   awk '{print $1, $3}' prints the first and third fields separated by space.

3. Final Answer:

   apple red banana yellow cherry red -> Option A

4. Quick Check:

   Print first and third fields [OK]

Hint: Match $1, $2, $3 to columns in order [OK]

Common Mistakes:

• Mixing up field numbers
• Printing wrong fields or all fields
• Ignoring spaces between printed fields

4. What is wrong with this command: awk '{print $4}' data.txt if data.txt has only 3 fields per line?

medium

A. It prints the third field instead

B. It causes a syntax error

C. It prints an empty line for each input line

D. It prints the entire line

Solution

1. Step 1: Understand field numbering in awk

   Fields are numbered from 1 to NF (number of fields). If $4 is requested but only 3 fields exist, $4 is empty.

2. Step 2: Predict output behavior

   Printing $4 prints nothing, resulting in empty lines for each input line.

3. Final Answer:

   It prints an empty line for each input line -> Option C

4. Quick Check:

   Missing field prints empty [OK]

Hint: Fields beyond NF print empty, no error [OK]

Common Mistakes:

• Expecting syntax error for missing fields
• Thinking it prints last field instead
• Assuming it prints whole line if field missing

5. You want to print only the lines from file.txt where the second field is greater than 100 using awk. Which command achieves this?

hard

A. awk '{if $2 > 100 print $0}' file.txt

B. awk '$2 > 100 {print $0}' file.txt

C. awk '{print $0 if $2 > 100}' file.txt

D. awk '$2 => 100 {print $0}' file.txt

Solution

1. Step 1: Understand condition syntax in awk

   Conditions go before the action block. The correct comparison operator is >, not =>.

2. Step 2: Check each option

   awk '$2 > 100 {print $0}' file.txt uses correct syntax: $2 > 100 {print $0}. awk '{if $2 > 100 print $0}' file.txt misses parentheses and braces. awk '{print $0 if $2 > 100}' file.txt has wrong syntax. awk '$2 => 100 {print $0}' file.txt uses invalid operator =>.

3. Final Answer:

   awk '$2 > 100 {print $0}' file.txt -> Option B

4. Quick Check:

   Condition before action, use > operator [OK]

Hint: Put condition before braces, use > not => [OK]

Common Mistakes:

• Using invalid comparison operators like =>
• Placing if inside braces without parentheses
• Omitting braces or condition syntax errors