Process substitution (<() and >()) in Bash Scripting

Introduction

Process substitution lets you use the output or input of a command as if it were a file. This helps connect commands easily without creating temporary files.

You want to compare outputs of two commands using diff without saving files.

You want to feed the output of a command as input to another command that expects a file.

You want to save the output of a command into another command that reads from a file.

You want to avoid creating temporary files when chaining commands.

You want to use commands that require file arguments but want to use dynamic data.

Syntax

Bash Scripting

command1 <(command2)
command1 >(command2)

The <() syntax runs command2 and provides its output as a file-like input to command1.

The >() syntax runs command2 and provides a file-like output to command1, so command1 writes to it.

Examples

Compare the list of files in two directories without creating temporary files.

Bash Scripting

diff <(ls dir1) <(ls dir2)

Send input.log content to tee, which writes to both standard output and to the grep command that filters 'error' lines into errors.log.

Bash Scripting

tee >(grep 'error' > errors.log) < input.log

Print 'Hello' by using process substitution as a file input to cat.

Bash Scripting

cat <(echo 'Hello')

Sort combined contents of file1 and file2 without creating a merged file.

Bash Scripting

sort <(cat file1) <(cat file2)

Sample Program

This script creates two directories with different files, lists their contents, then uses process substitution with diff to compare the file lists without creating temporary files. It prints the differences found.

Bash Scripting

#!/bin/bash

# List files in two directories and compare them
mkdir -p dir1 dir2

echo 'fileA' > dir1/fileA.txt
echo 'fileB' > dir1/fileB.txt

echo 'fileA' > dir2/fileA.txt
echo 'fileC' > dir2/fileC.txt

# Show files in dir1
ls dir1

# Show files in dir2
ls dir2

# Use process substitution to compare files

diff_output=$(diff <(ls dir1) <(ls dir2))

# Print diff result
if [ -z "$diff_output" ]; then
  echo "Directories have the same files."
else
  echo "Differences found:"
  echo "$diff_output"
fi

# Clean up
tmp_dirs=(dir1 dir2)
for d in "${tmp_dirs[@]}"; do
  rm -r "$d"
done

Important Notes

Process substitution uses named pipes or /dev/fd files internally, so it does not create real temporary files on disk.

Time complexity depends on the commands used inside the substitution, not on the substitution itself.

A common mistake is to forget that process substitution provides a filename, so you must use commands that accept filenames as arguments.

Use process substitution when you want to avoid temporary files and your commands support file arguments.

Summary

Process substitution lets you treat command output or input as files.

Use <() to provide command output as input file to another command.

Use >() to provide a writable file to a command that sends output to another command.

Practice

1. What does the process substitution syntax <() do in bash scripting?

easy

A. It runs a command and waits for user input.

B. It redirects the output of a command to a file on disk.

C. It creates a temporary file with command output saved.

D. It runs a command and provides its output as a file input to another command.

Solution

1. Step 1: Understand process substitution <()

   This syntax runs a command and makes its output appear as a file that another command can read.

2. Step 2: Compare with other options

   Unlike redirecting output to a disk file, <() creates a temporary file descriptor for input.

3. Final Answer:

   It runs a command and provides its output as a file input to another command. -> Option D

4. Quick Check:

   Process substitution <() = command output as input file [OK]

Hint: Remember: <() gives command output as a file input [OK]

Common Mistakes:

• Confusing <() with output redirection >
• Thinking it creates a real disk file
• Assuming it waits for user input

2. Which of the following is the correct syntax to use process substitution to compare two files file1 and file2 using diff?

easy

A. diff <(cat file1) <(cat file2)

B. diff >(cat file1) >(cat file2)

C. diff <(file1) <(file2)

D. diff file1 >() file2

Solution

1. Step 1: Identify correct process substitution usage

   Using <() runs commands and provides their output as files to diff.

2. Step 2: Check each option

   diff <(cat file1) <(cat file2) correctly uses cat file1 and cat file2 inside <(). Options B, C, and D misuse syntax or operators.

3. Final Answer:

   diff <(cat file1) <(cat file2) -> Option A

4. Quick Check:

   Correct syntax for process substitution in diff = diff <(cat file1) <(cat file2) [OK]

Hint: Use <() with commands producing input, not >() [OK]

Common Mistakes:

• Using >() instead of <() for input
• Trying to put filenames directly inside <()
• Mixing redirection symbols incorrectly

3. What will be the output of the following bash command?

comm -12 <(sort fileA.txt) <(sort fileB.txt)

Assuming fileA.txt contains:
apple banana carrot
and fileB.txt contains:
banana carrot date

medium

A. apple
banana
carrot
date

B. banana
carrot

C. apple

D. banana
carrot
date
apple

Solution

1. Step 1: Understand comm -12 with process substitution

   The command compares sorted files and outputs lines common to both (options 1 and 2).

2. Step 2: Sort and compare file contents

   Sorted fileA.txt: apple, banana, carrot
   Sorted fileB.txt: banana, carrot, date
   Common lines: banana, carrot

3. Final Answer:

   banana carrot -> Option B

4. Quick Check:

   Common sorted lines = banana and carrot [OK]

Hint: comm -12 shows common lines from sorted inputs [OK]

Common Mistakes:

• Not sorting files before comm
• Expecting all lines combined
• Confusing comm options

4. Identify the error in this script snippet using process substitution:

grep 'error' >(sort > errors_sorted.txt)

medium

A. Using >() where <() is needed for input

B. Missing quotes around 'error'

C. Cannot redirect output inside >()

D. sort command should be outside process substitution

Solution

1. Step 1: Analyze the use of >() in the command

   The syntax >() creates a writable file descriptor for output redirection, but here grep expects a file input, not output.

2. Step 2: Determine correct substitution type

   To provide input from a command, <() should be used. Using >() here is incorrect.

3. Final Answer:

   Using >() where <() is needed for input -> Option A

4. Quick Check:

   Input needs <() not >() [OK]

Hint: Use <() for input, >() for output redirection [OK]

Common Mistakes:

• Confusing input and output process substitution
• Assuming >() can be nested inside commands
• Ignoring command expectations for input/output

5. You want to run a command that reads from a file and writes output to another command simultaneously, without creating temporary files on disk. Which process substitution usage correctly achieves this?

command1 <(producer) >(consumer)

What does this setup do?

hard

A. Runs command1 output to producer and input from consumer.

B. Creates two temporary files for producer and consumer.

C. Runs producer and feeds its output as input to command1, then sends command1 output to consumer.

D. Runs consumer first, then producer.

Solution

1. Step 1: Understand <() and >() roles

   <() provides command output as input file, >() provides a writable file for output redirection.

2. Step 2: Analyze the command flow

   producer output feeds command1 input via <(), and command1 output is sent to consumer via >().

3. Final Answer:

   Runs producer and feeds its output as input to command1, then sends command1 output to consumer. -> Option C

4. Quick Check:

   <() input, >() output redirection [OK]

Hint: Use <() for input, >() for output to chain commands [OK]

Common Mistakes:

• Mixing input and output substitutions
• Assuming temporary disk files are created
• Confusing command execution order