Subprocess testing in PyTest - Build an Automation Script

Test subprocess call to list directory contents

Preconditions (3)

Step 1: Write a Python function that uses subprocess.run to execute 'ls' (Linux/macOS) or 'dir' (Windows) command

Step 2: Capture the output of the subprocess call

Step 3: Write a pytest test function to call this function

Step 4: Assert that the subprocess call returns exit code 0

Step 5: Assert that the output contains at least one known file or directory name (e.g., the current script file or folder)

✅ Expected Result: The test passes if the subprocess call succeeds with exit code 0 and the output contains the expected file or directory name.

Automation Requirements - pytest

Assertions Needed:

Assert subprocess returncode is 0

Assert output contains expected file or directory name

Best Practices:

Use subprocess.run with capture_output=True and text=True for easy output handling

Use pytest fixtures if needed for setup

Avoid hardcoding OS-specific commands; use platform detection

Use clear and descriptive assertion messages

Automated Solution

PyTest

import subprocess
import sys
import os
import pytest

def list_directory():
    if sys.platform.startswith('win'):
        cmd = ['cmd', '/c', 'dir']
    else:
        cmd = ['ls']
    result = subprocess.run(cmd, capture_output=True, text=True)
    return result


def test_list_directory_contains_this_file():
    result = list_directory()
    assert result.returncode == 0, f"Subprocess failed with code {result.returncode}"
    # Check that the output contains this test file name
    this_file = os.path.basename(__file__)
    assert this_file in result.stdout, f"Output does not contain expected file name '{this_file}'"

The list_directory function runs the system command to list directory contents. It detects the OS to choose the right command (dir for Windows, ls for others).

We use subprocess.run with capture_output=True and text=True to get the output as a string.

The test test_list_directory_contains_this_file calls this function, then asserts the return code is 0, meaning success.

It also asserts that the output contains the current test file name, ensuring the command listed the directory contents correctly.

Assertions include messages to help understand failures.

Common Mistakes - 3 Pitfalls

Using subprocess.call instead of subprocess.run

{'mistake': "Hardcoding 'ls' command without OS check", 'why_bad': "The 'ls' command does not work on Windows, causing test failures on that platform.", 'correct_approach': "Detect the OS using sys.platform and choose 'dir' for Windows and 'ls' for others."}

Not checking subprocess return code before using output

Bonus Challenge

Now add data-driven testing with 3 different commands: 'ls' or 'dir', 'echo Hello', and an invalid command to test failure.

Show Hint

Practice

1. What is the main purpose of subprocess testing in pytest?

easy

A. To measure the speed of your Python functions

B. To test user interface elements like buttons and forms

C. To check commands run by your program and verify their output

D. To check database connections and queries

Solution

1. Step 1: Understand subprocess testing

   Subprocess testing focuses on running external commands or programs from your code and checking their behavior.

2. Step 2: Identify the correct purpose

   Among the options, only checking commands run by your program matches subprocess testing.

3. Final Answer:

   To check commands run by your program and verify their output -> Option C

4. Quick Check:

   Subprocess testing = check commands run [OK]

Hint: Subprocess testing checks external commands run by your code [OK]

Common Mistakes:

• Confusing subprocess testing with UI testing
• Thinking it measures function speed
• Assuming it tests databases

2. Which of the following is the correct way to run a subprocess command in pytest and capture its output?

easy

A. subprocess.execute('ls', capture=True)

B. subprocess.run(['ls'], capture_output=True, text=True)

C. subprocess.call('ls', output=True)

D. subprocess.run('ls', capture=True)

Solution

1. Step 1: Recall subprocess.run syntax

   The correct function is subprocess.run with a list of command arguments and capture_output=True to capture output.

2. Step 2: Check options for correctness

   Only subprocess.run(['ls'], capture_output=True, text=True) uses subprocess.run with correct parameters and argument format.

3. Final Answer:

   subprocess.run(['ls'], capture_output=True, text=True) -> Option B

4. Quick Check:

   Use subprocess.run with capture_output=True [OK]

Hint: Use subprocess.run([...], capture_output=True) to capture output [OK]

Common Mistakes:

• Using subprocess.execute which does not exist
• Passing command as string without shell=True
• Using wrong parameter names like capture or output

3. Given the following pytest test code, what will be the output of the assertion?

import subprocess

def test_echo():
    result = subprocess.run(['echo', 'hello'], capture_output=True, text=True)
    assert result.stdout == 'hello\n'

medium

A. The test raises a runtime error

B. The test fails because stdout is empty

C. The test fails because stdout contains 'hello' without newline

D. The test passes because stdout contains 'hello\n'

Solution

1. Step 1: Understand subprocess.run output for echo

   The echo command outputs the string followed by a newline, so stdout will be 'hello\n'.

2. Step 2: Check the assertion

   The assertion compares result.stdout to 'hello\n', which matches exactly, so it passes.

3. Final Answer:

   The test passes because stdout contains 'hello\n' -> Option D

4. Quick Check:

   echo adds newline, assertion matches [OK]

Hint: Remember echo adds newline to output [OK]

Common Mistakes:

• Forgetting echo adds a newline
• Expecting stdout without newline
• Confusing stdout with stderr

4. Identify the error in this pytest subprocess test code:

import subprocess

def test_fail():
    result = subprocess.run(['false'], capture_output=True, text=True)
    assert result.returncode == 0

medium

A. The command 'false' returns a non-zero exit code, so assertion fails

B. Missing capture_output=True causes error

C. Using text=True is invalid here

D. subprocess.run requires shell=True for 'false'

Solution

1. Step 1: Understand the 'false' command behavior

   The 'false' command always returns exit code 1 (failure), not 0.

2. Step 2: Check the assertion on returncode

   The test asserts returncode == 0, which is false, so the assertion fails.

3. Final Answer:

   The command 'false' returns a non-zero exit code, so assertion fails -> Option A

4. Quick Check:

   'false' returns 1, assertion expects 0 [OK]

Hint: Check command return codes before asserting 0 [OK]

Common Mistakes:

• Assuming 'false' returns 0
• Thinking capture_output=True is mandatory for returncode
• Believing shell=True is needed for 'false'

5. You want to test a subprocess command that may output errors. Which pytest assertion correctly checks that the command failed and printed 'error' in stderr?

result = subprocess.run(['mycmd'], capture_output=True, text=True)

hard

A. assert result.returncode != 0 and 'error' in result.stderr

B. assert result.returncode == 0 and 'error' in result.stdout

C. assert result.returncode == 0 and 'error' in result.stderr

D. assert result.returncode != 0 and 'error' in result.stdout

Solution

1. Step 1: Understand failure and error output

   A failed command has returncode not zero and error messages appear in stderr.

2. Step 2: Match assertion to expected behavior

   assert result.returncode != 0 and 'error' in result.stderr asserts returncode != 0 and 'error' in stderr, which correctly tests failure and error output.

3. Final Answer:

   assert result.returncode != 0 and 'error' in result.stderr -> Option A

4. Quick Check:

   Failure means returncode != 0 and errors in stderr [OK]

Hint: Check returncode != 0 and error text in stderr [OK]

Common Mistakes:

• Checking error in stdout instead of stderr
• Expecting returncode == 0 for failure
• Mixing stdout and stderr in assertions