sed for substitution in scripts in Bash Scripting

Introduction

You use sed substitution to quickly find and replace text in files or outputs without opening them. It helps automate text changes in scripts.

Fix a typo in many lines of a configuration file automatically.

Change a word in a log file before sharing it.

Update a version number in multiple files during deployment.

Replace placeholders in a template file with real values.

Clean up output text by removing or changing certain words.

Syntax

Bash Scripting

sed 's/old-text/new-text/' filename

# or for inline replacement in a script
sed 's/old-text/new-text/g' <<< "$variable"

The 's' means substitute. It looks for 'old-text' and replaces it with 'new-text'.

Adding 'g' after the last slash means replace all occurrences in a line, not just the first.

Examples

Replaces the first 'cat' with 'dog' on each line of animals.txt.

Bash Scripting

sed 's/cat/dog/' animals.txt

Replaces all 'cat' words with 'dog' on each line.

Bash Scripting

sed 's/cat/dog/g' animals.txt

Replaces 'Hello' only if it is at the start of a line.

Bash Scripting

sed 's/^Hello/Hi/' greetings.txt

Replaces the first 'dog' with 'cat' in the given string.

Bash Scripting

sed 's/dog/cat/' <<< "I have a dog and a dog."

Sample Program

This script creates a file with some lines about cats. It then shows how sed replaces the first 'cat' per line and then all 'cat' words per line with 'dog'.

Bash Scripting

#!/bin/bash

# Create a sample file
cat <<EOF > pets.txt
I have a cat.
My cat is cute.
Cats are playful.
EOF

# Show original file
echo "Original file contents:"
cat pets.txt

echo
# Replace first 'cat' with 'dog' on each line
sed 's/cat/dog/' pets.txt > pets_replaced.txt

echo "After replacing first 'cat' with 'dog' on each line:"
cat pets_replaced.txt

echo
# Replace all 'cat' (case sensitive) with 'dog' on each line
sed 's/cat/dog/g' pets.txt > pets_replaced_all.txt

echo "After replacing all 'cat' with 'dog' on each line:"
cat pets_replaced_all.txt

Important Notes

Time complexity is generally linear to the size of the input text because sed reads line by line.

Space complexity is low since sed streams the input and does not load entire files into memory.

A common mistake is forgetting the 'g' flag, which only replaces the first match per line.

Use sed substitution when you want quick, scriptable text changes without opening editors.

Summary

sed substitution lets you find and replace text easily in scripts.

Use 's/old/new/' to replace the first match per line, add 'g' to replace all matches.

It is great for automating text edits in files or variables.

Practice

1. What does the sed command sed 's/cat/dog/' file.txt do?

easy

A. Prints lines containing 'cat' from file.txt

B. Replaces all occurrences of 'cat' with 'dog' in the entire file.txt

C. Replaces the first occurrence of 'cat' with 'dog' on each line of file.txt

D. Deletes all lines containing 'cat' from file.txt

Solution

1. Step 1: Understand the sed substitution syntax

   The command sed 's/cat/dog/' file.txt uses s/old/new/ which replaces only the first match per line.

2. Step 2: Apply to the file content

   Each line in file.txt will have its first 'cat' replaced by 'dog', other occurrences remain unchanged.

3. Final Answer:

   Replaces the first occurrence of 'cat' with 'dog' on each line of file.txt -> Option C

4. Quick Check:

   sed s/old/new/ replaces first match per line [OK]

Hint: Remember: no 'g' means only first match replaced per line [OK]

Common Mistakes:

• Thinking it replaces all matches without 'g'
• Confusing substitution with deletion
• Assuming it prints only matching lines

2. Which of the following is the correct syntax to replace all occurrences of 'apple' with 'orange' in a file using sed?

easy

A. sed 's/apple/orange/' file.txt

B. sed 'replace apple with orange' file.txt

C. sed 's/apple/orange//g' file.txt

D. sed 's/apple/orange/g' file.txt

Solution

1. Step 1: Identify the global flag usage

   To replace all occurrences on each line, the 'g' flag is needed after the substitution pattern.

2. Step 2: Check each option

   sed 's/apple/orange/g' file.txt uses s/apple/orange/g which correctly replaces all 'apple' with 'orange'. sed 's/apple/orange/' file.txt replaces only first occurrence. sed 'replace apple with orange' file.txt is invalid syntax. sed 's/apple/orange//g' file.txt has extra slashes causing syntax error.

3. Final Answer:

   sed 's/apple/orange/g' file.txt -> Option D

4. Quick Check:

   Use 'g' flag for all matches [OK]

Hint: Add 'g' after last slash to replace all matches [OK]

Common Mistakes:

• Omitting the 'g' flag to replace all matches
• Adding extra slashes causing syntax errors
• Using incorrect command words like 'replace'

3. What will be the output of the following command if the input file contains the line apple apple apple?

sed 's/apple/orange/' file.txt

medium

A. orange orange orange

B. orange apple apple

C. apple orange apple

D. apple apple orange

Solution

1. Step 1: Understand sed substitution without 'g'

   Without the 'g' flag, sed replaces only the first occurrence of 'apple' per line.

2. Step 2: Apply substitution to the input line

   The first 'apple' becomes 'orange', the rest remain unchanged, resulting in 'orange apple apple'.

3. Final Answer:

   orange apple apple -> Option B

4. Quick Check:

   First match replaced only without 'g' [OK]

Hint: No 'g' means only first match changes per line [OK]

Common Mistakes:

• Assuming all occurrences change without 'g'
• Confusing order of words after substitution
• Thinking sed changes input file by default

4. You wrote this sed command to replace 'foo' with 'bar' globally: sed 's/foo/bar/' file.txt. But only the first 'foo' per line is replaced. What is the fix?

medium

A. Add the 'g' flag: sed 's/foo/bar/g' file.txt

B. Use double quotes instead of single quotes

C. Add a space after 'bar/'

D. Use sed -i option to edit in place

Solution

1. Step 1: Identify missing global flag

   The command lacks the 'g' flag, so only first occurrence per line is replaced.

2. Step 2: Add the 'g' flag to fix

   Adding 'g' after the substitution pattern makes sed replace all occurrences per line.

3. Final Answer:

   Add the 'g' flag: sed 's/foo/bar/g' file.txt -> Option A

4. Quick Check:

   Global flag 'g' replaces all matches [OK]

Hint: Add 'g' to replace all matches, not just first [OK]

Common Mistakes:

• Thinking quotes affect substitution behavior
• Adding spaces inside substitution pattern
• Confusing in-place edit with substitution flags

5. You want to replace the word 'error' with 'warning' only on lines containing 'log' in a file. Which sed command achieves this?

hard

A. sed '/log/ s/error/warning/g' file.txt

B. sed 's/error/warning/g /log/' file.txt

C. sed 's/error/warning/g if /log/' file.txt

D. sed 's/error/warning/g' file.txt | grep log

Solution

1. Step 1: Understand conditional substitution in sed

   To apply substitution only on lines matching a pattern, use /pattern/ s/old/new/flags syntax.

2. Step 2: Analyze options

   sed '/log/ s/error/warning/g' file.txt correctly applies substitution only on lines containing 'log'. Options A and C have invalid syntax. sed 's/error/warning/g' file.txt | grep log replaces everywhere then filters lines, which does not limit substitution.

3. Final Answer:

   sed '/log/ s/error/warning/g' file.txt -> Option A

4. Quick Check:

   Use address before s/// to limit lines [OK]

Hint: Put pattern before s/// to restrict substitution lines [OK]

Common Mistakes:

• Placing substitution after pattern incorrectly
• Trying to use if statements inside sed
• Filtering lines after substitution instead of limiting it