Testpaths configuration in PyTest - Build an Automation Script

Verify pytest only runs tests from specified testpaths

Preconditions (2)

Step 1: Create a pytest.ini file with testpaths set to 'tests/unit' directory

Step 2: Place test files in 'tests/unit' and 'tests/integration' directories

Step 3: Run pytest from the project root directory

Step 4: Observe which tests are executed

✅ Expected Result: pytest runs only the tests located in the 'tests/unit' directory and ignores tests in 'tests/integration'

Automation Requirements - pytest

Assertions Needed:

Verify that tests from 'tests/unit' directory are executed

Verify that tests from 'tests/integration' directory are not executed

Best Practices:

Use pytest's capsys or caplog to capture test output

Use temporary directories or fixtures to isolate test files

Use clear and descriptive test function names

Automated Solution

PyTest

import subprocess
import os
import sys
import tempfile
import shutil

def test_pytest_runs_only_testpaths():
    # Create a temporary directory to simulate project structure
    temp_dir = tempfile.mkdtemp()
    try:
        # Create 'tests/unit' and 'tests/integration' directories
        unit_dir = os.path.join(temp_dir, 'tests', 'unit')
        integration_dir = os.path.join(temp_dir, 'tests', 'integration')
        os.makedirs(unit_dir)
        os.makedirs(integration_dir)

        # Create a test file in 'tests/unit'
        unit_test_file = os.path.join(unit_dir, 'test_unit_sample.py')
        with open(unit_test_file, 'w') as f:
            f.write('def test_unit_pass():\n    assert True\n')

        # Create a test file in 'tests/integration'
        integration_test_file = os.path.join(integration_dir, 'test_integration_sample.py')
        with open(integration_test_file, 'w') as f:
            f.write('def test_integration_pass():\n    assert True\n')

        # Create pytest.ini with testpaths set to 'tests/unit'
        pytest_ini = os.path.join(temp_dir, 'pytest.ini')
        with open(pytest_ini, 'w') as f:
            f.write('[pytest]\ntestpaths = tests/unit\n')

        # Run pytest in the temp_dir
        result = subprocess.run(
            [sys.executable, '-m', 'pytest', '--tb=short', '-q'],
            cwd=temp_dir,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True
        )

        # Check that test from 'tests/unit' ran
        assert 'test_unit_pass' in result.stdout, 'Unit test did not run as expected'

        # Check that test from 'tests/integration' did NOT run
        assert 'test_integration_pass' not in result.stdout, 'Integration test should not run'

        # Check pytest exit code is 0 (success)
        assert result.returncode == 0, f'Pytest failed with exit code {result.returncode}'

    finally:
        shutil.rmtree(temp_dir)

This test creates a temporary project structure with two test directories: tests/unit and tests/integration. It writes one simple passing test in each directory.

It then creates a pytest.ini file in the root of this temporary project, configuring testpaths = tests/unit. This tells pytest to only look for tests inside tests/unit.

The test runs pytest as a subprocess in the temporary directory and captures the output.

Assertions check that the test from tests/unit was run (its name appears in output), and the test from tests/integration was not run (its name does not appear). It also checks that pytest exited successfully.

This approach uses temporary directories to avoid affecting the real project and subprocess to run pytest as a user would. It verifies the testpaths configuration works as expected.

Common Mistakes - 4 Pitfalls

Not creating a pytest.ini file with testpaths configured

Placing test files only in one directory and expecting pytest to ignore others

Using print statements instead of assertions to verify test execution

Running pytest without capturing output

Bonus Challenge

Now add data-driven testing with 3 different testpaths configurations and verify pytest runs tests only from those paths each time

Show Hint

Practice

1. What is the main purpose of the testpaths setting in a pytest.ini file?

easy

A. To configure the test report format

B. To specify which folders pytest should look in for test files

C. To set the timeout for each test

D. To define the order in which tests run

Solution

1. Step 1: Understand the role of testpaths

   The testpaths setting tells pytest where to find test files by listing directories.

2. Step 2: Compare with other options

   Other options like test order, timeout, or report format are configured differently, not with testpaths.

3. Final Answer:

   To specify which folders pytest should look in for test files -> Option B

4. Quick Check:

   testpaths = test folders [OK]

Hint: testpaths lists folders to find tests [OK]

Common Mistakes:

• Confusing testpaths with test order settings
• Thinking testpaths sets test timeouts
• Assuming testpaths changes report style

2. Which of the following is the correct syntax to set testpaths in pytest.ini to look in folders tests and integration?

easy

A. [pytest]
testpaths = tests integration

B. [pytest]
testpaths = ['tests', 'integration']

C. [pytest]
testpaths = tests,integration

D. [pytest]
testpaths = "tests integration"

Solution

1. Step 1: Check pytest.ini syntax for testpaths

   In pytest.ini, testpaths lists folders separated by spaces without quotes or commas.

2. Step 2: Validate each option

   [pytest] testpaths = tests integration uses space-separated folder names correctly. Options A and D use quotes or brackets incorrectly. [pytest] testpaths = tests,integration uses commas which pytest does not accept here.

3. Final Answer:

   [pytest] testpaths = tests integration -> Option A

4. Quick Check:

   Space-separated folder names, no quotes [OK]

Hint: Use space-separated folder names without quotes [OK]

Common Mistakes:

• Using commas instead of spaces
• Adding quotes or brackets around folder names
• Placing testpaths outside [pytest] section

3. Given this pytest.ini content:

[pytest]
testpaths = tests integration

And this folder structure:

project/
  tests/
    test_alpha.py
  integration/
    test_beta.py
  docs/
    test_docs.py

Which test files will pytest discover and run?

medium

A. Only test_docs.py

B. test_alpha.py, test_beta.py, and test_docs.py

C. Only test_alpha.py

D. test_alpha.py and test_beta.py only

Solution

1. Step 1: Identify folders listed in testpaths

   The testpaths setting includes tests and integration folders only.

2. Step 2: Match test files in those folders

   Files test_alpha.py in tests and test_beta.py in integration will be found. test_docs.py is in docs, which is not listed.

3. Final Answer:

   test_alpha.py and test_beta.py only -> Option D

4. Quick Check:

   Only testpaths folders' tests run [OK]

Hint: Only tests in testpaths folders run [OK]

Common Mistakes:

• Assuming all test files in project run
• Including tests from folders not in testpaths
• Confusing folder names with test file names

4. You set testpaths = tests in pytest.ini but pytest still discovers tests in other folders. What is the most likely cause?

medium

A. There is another config file overriding testpaths

B. You forgot to restart pytest after editing pytest.ini

C. testpaths only filters test names, not folders

D. pytest ignores testpaths by default

Solution

1. Step 1: Understand pytest config precedence

   pytest can read multiple config files like tox.ini or setup.cfg that may override pytest.ini.

2. Step 2: Check for conflicting configs

   If tests outside tests run, likely another config file sets different testpaths or disables it.

3. Final Answer:

   There is another config file overriding testpaths -> Option A

4. Quick Check:

   Multiple configs can override testpaths [OK]

Hint: Check for multiple config files overriding settings [OK]

Common Mistakes:

• Assuming pytest.ini changes apply without reload
• Thinking testpaths filters test names only
• Believing pytest ignores testpaths by default

5. You want pytest to only run tests in unit_tests and api_tests folders, but exclude api_tests/slow. How can you configure pytest.ini using testpaths and other settings to achieve this?

hard

A. [pytest]
testpaths = unit_tests
norecursedirs = api_tests/slow

B. [pytest]
testpaths = unit_tests api_tests/slow
norecursedirs = api_tests

C. [pytest]
testpaths = unit_tests api_tests
norecursedirs = api_tests/slow

D. [pytest]
testpaths = unit_tests api_tests
ignore = api_tests/slow

Solution

1. Step 1: Use testpaths to include desired folders

   Set testpaths = unit_tests api_tests to include both folders.

2. Step 2: Exclude the slow subfolder with norecursedirs

   Use norecursedirs = api_tests/slow to prevent pytest from searching inside that subfolder.

3. Step 3: Verify other options are incorrect

   [pytest] testpaths = unit_tests api_tests/slow norecursedirs = api_tests includes the slow folder in testpaths, which is wrong. [pytest] testpaths = unit_tests norecursedirs = api_tests/slow excludes api_tests entirely. [pytest] testpaths = unit_tests api_tests ignore = api_tests/slow uses ignore which is not a valid pytest.ini setting.

4. Final Answer:

   [pytest] testpaths = unit_tests api_tests norecursedirs = api_tests/slow -> Option C

5. Quick Check:

   Include folders with testpaths, exclude subfolders with norecursedirs [OK]

Hint: Use testpaths to include, norecursedirs to exclude subfolders [OK]

Common Mistakes:

• Including excluded folders in testpaths
• Using invalid config keys like ignore
• Excluding whole folders instead of subfolders