awk field extraction in scripts in Bash Scripting - Mini Project: Build & Apply

Extracting Fields with awk in Bash Scripts

📖 Scenario: You have a text file with information about employees. Each line contains the employee's ID, name, and department separated by spaces.You want to write a bash script that extracts specific fields from each line using awk.

🎯 Goal: Build a bash script that reads the employee data and extracts the employee names using awk.

📋 What You'll Learn

Create a text file variable with employee data

Create a variable for the field number to extract

Use awk to extract the specified field from each line

Print the extracted fields

💡 Why This Matters

🌍 Real World

Extracting specific columns from text data is common when processing logs, reports, or CSV files in automation scripts.

💼 Career

Knowing how to use awk in bash scripts helps automate data extraction tasks in system administration, data analysis, and DevOps roles.

1

Create the employee data variable

Create a variable called employee_data that contains these exact three lines of text, each line separated by a newline character:
101 Alice Sales
102 Bob Marketing
103 Charlie IT

Bash Scripting

# Create the variable employee_data with the exact lines
# Your code here

Hint

Use double quotes and \n to separate lines inside the string.

2

Set the field number to extract

Create a variable called field_number and set it to 2 to represent the employee name field.

Bash Scripting

employee_data="101 Alice Sales\n102 Bob Marketing\n103 Charlie IT"
# Create the variable field_number and set it to 2
# Your code here

Hint

Just assign the number 2 to the variable field_number.

3

Extract the specified field using awk

Use echo to send employee_data to awk. Use awk with the field variable field_number to extract that field from each line. Store the result in a variable called extracted_names.

Bash Scripting

employee_data="101 Alice Sales\n102 Bob Marketing\n103 Charlie IT"
field_number=2
# Use awk to extract the field_number field and save to extracted_names
# Your code here

Hint

Use awk -v field="$field_number" '{print $field}' to print the chosen field.

4

Print the extracted names

Print the variable extracted_names to display the extracted employee names.

Bash Scripting

employee_data="101 Alice Sales\n102 Bob Marketing\n103 Charlie IT"
field_number=2
extracted_names=$(echo -e "$employee_data" | awk -v field="$field_number" '{print $field}')
# Print the extracted_names variable
# Your code here

Hint

Use printf "%s\n" "$extracted_names" to print each name on its own line.

Practice

1. What does awk '{print $2}' file.txt do in a bash script?

easy

A. Prints the second word (field) from each line of file.txt

B. Prints the entire line from file.txt

C. Prints the first word (field) from each line of file.txt

D. Prints the number of lines in file.txt

Solution

1. Step 1: Understand awk field numbering

   Awk splits each line into fields separated by spaces by default. $1 is the first field, $2 is the second, and so on.

2. Step 2: Analyze the command

   The command prints $2, which is the second field of each line in file.txt.

3. Final Answer:

   Prints the second word (field) from each line of file.txt -> Option A

4. Quick Check:

   Awk $2 prints second field [OK]

Hint: Remember $1 is first field, $2 second, etc. [OK]

Common Mistakes:

• Confusing $2 with line number
• Thinking it prints entire line
• Mixing up field numbers starting at 0

2. Which of the following is the correct syntax to print the third field using awk with a comma as the field separator?

easy

A. awk -F ',' '{print $3}' file.csv

B. awk '{print $3 -F ','}' file.csv

C. awk '{print $3, -F ","}' file.csv

D. awk -F ';' '{print $3}' file.csv

Solution

1. Step 1: Identify field separator option

   The -F option sets the field separator. For comma-separated files, use -F ','.

2. Step 2: Check print syntax

   The correct print syntax is '{print $3}' to print the third field.

3. Final Answer:

   awk -F ',' '{print $3}' file.csv -> Option A

4. Quick Check:

   Use -F ',' before script block [OK]

Hint: Use -F 'separator' before the awk script block [OK]

Common Mistakes:

• Placing -F inside the print statement
• Using wrong separator like semicolon for CSV
• Misplacing quotes around separator

3. Given a file data.txt with the following content:

apple 10 red
banana 5 yellow
cherry 20 red

What is the output of awk '{print $1, $3}' data.txt?

medium

A. apple 10
banana 5
cherry 20

B. apple red
banana yellow
cherry red

C. 10 red
5 yellow
20 red

D. apple banana cherry

Solution

1. Step 1: Identify fields in each line

   Each line has three fields: $1=fruit, $2=number, $3=color.

2. Step 2: Print first and third fields

   The command prints $1 and $3 separated by space, so output lines are fruit and color pairs.

3. Final Answer:

   apple red banana yellow cherry red -> Option B

4. Quick Check:

   Print $1 and $3 fields [OK]

Hint: Match $1, $2, $3 to columns in data [OK]

Common Mistakes:

• Printing wrong fields like $2 instead of $3
• Confusing field order
• Expecting numbers instead of colors

4. What is wrong with this awk command to print the first field from a colon-separated file?
awk '{print $1}' -F ':' file.txt

medium

A. The colon separator is invalid in awk

B. The $1 should be $0 to print the whole line

C. The -F option should come before the script block

D. The file name should come before the script block

Solution

1. Step 1: Check placement of -F option

   The -F option must come before the script block in awk commands.

2. Step 2: Analyze the command structure

   Correct syntax is: awk -F ':' '{print $1}' file.txt

3. Final Answer:

   The -F option should come before the script block -> Option C

4. Quick Check:

   -F before script block [OK]

Hint: Put -F before the '{print ...}' block [OK]

Common Mistakes:

• Placing -F after the script block
• Using wrong field variable
• Misordering file name and script

5. You have a file records.txt with lines like:

John|Doe|30|Engineer
Jane|Smith|25|Designer
Bob|Brown|40|Manager

Write an awk command to print only the names (first and last) of people older than 28.

hard

A. awk -F '|' '$3 < 28 {print $1, $2}' records.txt

B. awk -F ',' '$3 > 28 {print $1, $2}' records.txt

C. awk '{if ($3 > 28) print $1, $2}' records.txt

D. awk -F '|' '$3 > 28 {print $1, $2}' records.txt

Solution

1. Step 1: Set correct field separator

   The file uses '|' as separator, so use -F '|' to split fields correctly.

2. Step 2: Filter by age and print names

   Use condition $3 > 28 to select lines where age is greater than 28, then print first and second fields (first and last names).

3. Final Answer:

   awk -F '|' '$3 > 28 {print $1, $2}' records.txt -> Option D

4. Quick Check:

   Filter $3 > 28 and print $1, $2 [OK]

Hint: Use -F '|' and condition $3 > 28 to filter [OK]

Common Mistakes:

• Using wrong field separator
• Using wrong comparison operator
• Not filtering before printing