Verify pytest only runs tests from specified testpaths

Preconditions (2)

Step 1: Create a pytest.ini file with testpaths set to 'tests/unit' directory

Step 2: Place test files in 'tests/unit' and 'tests/integration' directories

Step 3: Run pytest from the project root directory

Step 4: Observe which tests are executed

✅ Expected Result: pytest runs only the tests located in the 'tests/unit' directory and ignores tests in 'tests/integration'

Automation Requirements - pytest

Assertions Needed:

Verify that tests from 'tests/unit' directory are executed

Verify that tests from 'tests/integration' directory are not executed

Best Practices:

Use pytest's capsys or caplog to capture test output

Use temporary directories or fixtures to isolate test files

Use clear and descriptive test function names

Automated Solution

PyTest

import subprocess
import os
import sys
import tempfile
import shutil

def test_pytest_runs_only_testpaths():
    # Create a temporary directory to simulate project structure
    temp_dir = tempfile.mkdtemp()
    try:
        # Create 'tests/unit' and 'tests/integration' directories
        unit_dir = os.path.join(temp_dir, 'tests', 'unit')
        integration_dir = os.path.join(temp_dir, 'tests', 'integration')
        os.makedirs(unit_dir)
        os.makedirs(integration_dir)

        # Create a test file in 'tests/unit'
        unit_test_file = os.path.join(unit_dir, 'test_unit_sample.py')
        with open(unit_test_file, 'w') as f:
            f.write('def test_unit_pass():\n    assert True\n')

        # Create a test file in 'tests/integration'
        integration_test_file = os.path.join(integration_dir, 'test_integration_sample.py')
        with open(integration_test_file, 'w') as f:
            f.write('def test_integration_pass():\n    assert True\n')

        # Create pytest.ini with testpaths set to 'tests/unit'
        pytest_ini = os.path.join(temp_dir, 'pytest.ini')
        with open(pytest_ini, 'w') as f:
            f.write('[pytest]\ntestpaths = tests/unit\n')

        # Run pytest in the temp_dir
        result = subprocess.run(
            [sys.executable, '-m', 'pytest', '--tb=short', '-q'],
            cwd=temp_dir,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True
        )

        # Check that test from 'tests/unit' ran
        assert 'test_unit_pass' in result.stdout, 'Unit test did not run as expected'

        # Check that test from 'tests/integration' did NOT run
        assert 'test_integration_pass' not in result.stdout, 'Integration test should not run'

        # Check pytest exit code is 0 (success)
        assert result.returncode == 0, f'Pytest failed with exit code {result.returncode}'

    finally:
        shutil.rmtree(temp_dir)

This test creates a temporary project structure with two test directories: tests/unit and tests/integration. It writes one simple passing test in each directory.

It then creates a pytest.ini file in the root of this temporary project, configuring testpaths = tests/unit. This tells pytest to only look for tests inside tests/unit.

The test runs pytest as a subprocess in the temporary directory and captures the output.

Assertions check that the test from tests/unit was run (its name appears in output), and the test from tests/integration was not run (its name does not appear). It also checks that pytest exited successfully.

This approach uses temporary directories to avoid affecting the real project and subprocess to run pytest as a user would. It verifies the testpaths configuration works as expected.

Common Mistakes - 4 Pitfalls

Not creating a pytest.ini file with testpaths configured

Placing test files only in one directory and expecting pytest to ignore others

Using print statements instead of assertions to verify test execution

Running pytest without capturing output

Bonus Challenge

Now add data-driven testing with 3 different testpaths configurations and verify pytest runs tests only from those paths each time

Show Hint