Process substitution (<() and >()) in Bash Scripting - Step-by-Step Execution

Concept Flow - Process substitution (<() and >())

Start command needing file input/output

↓

Use <() or >() to create temp FIFO or /dev/fd

↓

Run inner command inside <() or >()

↓

Shell connects FIFO or file descriptor to outer command

↓

Outer command reads from or writes to this stream

↓

Process substitution ends when commands finish

Process substitution lets a command use the output or input of another command as if it were a file, using special temporary streams.

Execution Sample

Bash Scripting

diff <(echo "apple") <(echo "orange")

Compares outputs of two echo commands as if they were files using process substitution.

Execution Table

Step	Action	Command/Subshell	File Descriptor/FIFO Created	Outer Command Input	Output Produced
1	Start diff command	diff	None yet	Waiting for inputs	None
2	Create process substitution for first <()	echo "apple"	/dev/fd/63 (or FIFO)	diff reads from /dev/fd/63	Outputs 'apple'
3	Create process substitution for second <()	echo "orange"	/dev/fd/62 (or FIFO)	diff reads from /dev/fd/62	Outputs 'orange'
4	diff compares inputs	diff	Reads from both descriptors	Reads 'apple' and 'orange'	Outputs diff lines
5	diff outputs result	diff	No new descriptors	N/A	1c1 < apple --- > orange
6	Commands finish	All	Descriptors closed	No input	Process substitution ends

💡 Process substitution ends after both inner commands finish and diff completes comparison.

Variable Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	Final
/dev/fd descriptors	None	/dev/fd/63 created for first echo	/dev/fd/62 created for second echo	Both descriptors open for diff	Descriptors closed
diff input streams	None	Reads from /dev/fd/63	Reads from /dev/fd/62	Reads both streams	No input
diff output	None	None	None	Diff result lines	Output complete

Key Moments - 3 Insights

Why does diff treat <(echo "apple") like a file?

Because process substitution creates a temporary file descriptor or FIFO that looks like a file, so diff can read from it as if it were a normal file (see execution_table steps 2 and 3).

What happens if the inner command in <() runs slowly?

The outer command (diff) waits for input from the file descriptor until the inner command produces output or finishes, so the process substitution streams data as it becomes available (see execution_table step 4).

Can process substitution be used for output redirection with >()?

Yes, >() creates a writable file descriptor that the outer command writes to, and the inner command reads from that stream (not shown here but works similarly to <()).

Visual Quiz - 3 Questions

Look at the execution_table at step 3, what is created for the second process substitution?

A /dev/fd/62 or a FIFO

B /dev/fd/63 or a FIFO

C A regular file on disk

D No file descriptor is created

💡 Hint

Check the 'File Descriptor/FIFO Created' column at step 3 in the execution_table.

At which step does diff start reading both inputs simultaneously?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Look for when diff reads from both descriptors in the 'Outer Command Input' column.

If the first echo command output changed to "banana", what would change in the execution_table output?

A The process substitution would fail

B The file descriptor numbers would change

C The diff output lines would show 'banana' instead of 'apple'

D No change in output

💡 Hint

Check the 'Output Produced' column at step 5 and consider what diff compares.

Concept Snapshot

Process substitution lets commands use outputs or inputs of other commands as files.
Syntax: <(command) for input, >(command) for output.
Shell creates temporary file descriptors or FIFOs.
Outer command reads/writes these like files.
Useful for commands needing file arguments but you want to use command output directly.

Full Transcript

Process substitution in bash scripting allows a command to use the output or input of another command as if it were a file. When you write diff <(echo "apple") <(echo "orange"), the shell runs the echo commands inside temporary file descriptors or FIFOs. These act like files for diff to read. The shell connects these streams so diff can compare the outputs. The process substitution ends when all commands finish. This lets you avoid creating real temporary files and makes scripts cleaner and faster.

Practice

1. What does the process substitution syntax <() do in bash scripting?

easy

A. It runs a command and waits for user input.

B. It redirects the output of a command to a file on disk.

C. It creates a temporary file with command output saved.

D. It runs a command and provides its output as a file input to another command.

Solution

1. Step 1: Understand process substitution <()

   This syntax runs a command and makes its output appear as a file that another command can read.

2. Step 2: Compare with other options

   Unlike redirecting output to a disk file, <() creates a temporary file descriptor for input.

3. Final Answer:

   It runs a command and provides its output as a file input to another command. -> Option D

4. Quick Check:

   Process substitution <() = command output as input file [OK]

Hint: Remember: <() gives command output as a file input [OK]

Common Mistakes:

• Confusing <() with output redirection >
• Thinking it creates a real disk file
• Assuming it waits for user input

2. Which of the following is the correct syntax to use process substitution to compare two files file1 and file2 using diff?

easy

A. diff <(cat file1) <(cat file2)

B. diff >(cat file1) >(cat file2)

C. diff <(file1) <(file2)

D. diff file1 >() file2

Solution

1. Step 1: Identify correct process substitution usage

   Using <() runs commands and provides their output as files to diff.

2. Step 2: Check each option

   diff <(cat file1) <(cat file2) correctly uses cat file1 and cat file2 inside <(). Options B, C, and D misuse syntax or operators.

3. Final Answer:

   diff <(cat file1) <(cat file2) -> Option A

4. Quick Check:

   Correct syntax for process substitution in diff = diff <(cat file1) <(cat file2) [OK]

Hint: Use <() with commands producing input, not >() [OK]

Common Mistakes:

• Using >() instead of <() for input
• Trying to put filenames directly inside <()
• Mixing redirection symbols incorrectly

3. What will be the output of the following bash command?

comm -12 <(sort fileA.txt) <(sort fileB.txt)

Assuming fileA.txt contains:
apple banana carrot
and fileB.txt contains:
banana carrot date

medium

A. apple
banana
carrot
date

B. banana
carrot

C. apple

D. banana
carrot
date
apple

Solution

1. Step 1: Understand comm -12 with process substitution

   The command compares sorted files and outputs lines common to both (options 1 and 2).

2. Step 2: Sort and compare file contents

   Sorted fileA.txt: apple, banana, carrot
   Sorted fileB.txt: banana, carrot, date
   Common lines: banana, carrot

3. Final Answer:

   banana carrot -> Option B

4. Quick Check:

   Common sorted lines = banana and carrot [OK]

Hint: comm -12 shows common lines from sorted inputs [OK]

Common Mistakes:

• Not sorting files before comm
• Expecting all lines combined
• Confusing comm options

4. Identify the error in this script snippet using process substitution:

grep 'error' >(sort > errors_sorted.txt)

medium

A. Using >() where <() is needed for input

B. Missing quotes around 'error'

C. Cannot redirect output inside >()

D. sort command should be outside process substitution

Solution

1. Step 1: Analyze the use of >() in the command

   The syntax >() creates a writable file descriptor for output redirection, but here grep expects a file input, not output.

2. Step 2: Determine correct substitution type

   To provide input from a command, <() should be used. Using >() here is incorrect.

3. Final Answer:

   Using >() where <() is needed for input -> Option A

4. Quick Check:

   Input needs <() not >() [OK]

Hint: Use <() for input, >() for output redirection [OK]

Common Mistakes:

• Confusing input and output process substitution
• Assuming >() can be nested inside commands
• Ignoring command expectations for input/output

5. You want to run a command that reads from a file and writes output to another command simultaneously, without creating temporary files on disk. Which process substitution usage correctly achieves this?

command1 <(producer) >(consumer)

What does this setup do?

hard

A. Runs command1 output to producer and input from consumer.

B. Creates two temporary files for producer and consumer.

C. Runs producer and feeds its output as input to command1, then sends command1 output to consumer.

D. Runs consumer first, then producer.

Solution

1. Step 1: Understand <() and >() roles

   <() provides command output as input file, >() provides a writable file for output redirection.

2. Step 2: Analyze the command flow

   producer output feeds command1 input via <(), and command1 output is sent to consumer via >().

3. Final Answer:

   Runs producer and feeds its output as input to command1, then sends command1 output to consumer. -> Option C

4. Quick Check:

   <() input, >() output redirection [OK]

Hint: Use <() for input, >() for output to chain commands [OK]

Common Mistakes:

• Mixing input and output substitutions
• Assuming temporary disk files are created
• Confusing command execution order