Verify pytest discovers test files and test functions correctly

Preconditions (2)

Step 1: Create a Python file named test_sample.py with two test functions: test_addition and test_subtraction

Step 2: Create another Python file named sample_test.py with one test function: test_multiplication

Step 3: Create a Python file named sample.py with one function named test_division (not a test file)

Step 4: Run pytest from the project root folder without any arguments

Step 5: Observe which tests pytest discovers and runs

✅ Expected Result: pytest discovers and runs test_addition and test_subtraction from test_sample.py and test_multiplication from sample_test.py. It does not run test_division from sample.py because the file name does not match pytest discovery rules.

Automation Requirements - pytest

Assertions Needed:

Verify that pytest collects exactly three tests

Verify that the test functions test_addition, test_subtraction, and test_multiplication are run

Verify that test_division is not collected or run

Best Practices:

Use pytest's built-in test discovery without custom arguments

Name test files and test functions following pytest conventions (test_*.py or *_test.py for files, test_* for functions)

Use assert statements for validation

Keep test functions independent and simple

Automated Solution

PyTest

import pytest
import os
import subprocess
import sys

def write_file(filename: str, content: str):
    with open(filename, 'w') as f:
        f.write(content)

def test_pytest_discovery(tmp_path):
    # Create test_sample.py with two test functions
    test_sample_content = '''
def test_addition():
    assert 1 + 1 == 2

def test_subtraction():
    assert 2 - 1 == 1
'''
    write_file(tmp_path / 'test_sample.py', test_sample_content)

    # Create sample_test.py with one test function
    sample_test_content = '''
def test_multiplication():
    assert 2 * 3 == 6
'''
    write_file(tmp_path / 'sample_test.py', sample_test_content)

    # Create sample.py with a function named test_division (not a test file)
    sample_content = '''
def test_division():
    assert 6 / 2 == 3
'''
    write_file(tmp_path / 'sample.py', sample_content)

    # Run pytest in the tmp_path directory
    result = subprocess.run(
        [sys.executable, '-m', 'pytest', '--collect-only', '-q'],
        cwd=tmp_path,
        capture_output=True,
        text=True
    )

    # The output lists collected tests by their node ids
    collected_tests = result.stdout.strip().split('\n')

    # We expect exactly 3 tests collected
    assert len(collected_tests) == 3, f"Expected 3 tests collected, got {len(collected_tests)}"

    # Check that the collected tests are the expected ones
    expected_tests = {
        'test_sample.py::test_addition',
        'test_sample.py::test_subtraction',
        'sample_test.py::test_multiplication'
    }

    assert set(collected_tests) == expected_tests, f"Collected tests do not match expected. Got: {collected_tests}"

    # Run pytest normally and capture output
    run_result = subprocess.run(
        [sys.executable, '-m', 'pytest', '-q', '--tb=no'],
        cwd=tmp_path,
        capture_output=True,
        text=True
    )

    # Check that the output contains the names of the three tests run
    for test_name in ['test_addition', 'test_subtraction', 'test_multiplication']:
        assert test_name in run_result.stdout, f"{test_name} not found in pytest output"

    # Check that test_division is not run
    assert 'test_division' not in run_result.stdout, "test_division should not be run because file name does not match discovery rules"

This test script uses pytest and Python's subprocess module to verify pytest's test discovery rules.

First, it creates three Python files in a temporary directory:

test_sample.py with two test functions.
sample_test.py with one test function.
sample.py with a function named like a test but in a non-test file.

Then, it runs pytest --collect-only to list all discovered tests and asserts that only the three functions in the correctly named test files are collected.

Next, it runs pytest normally to execute the tests and checks the output to confirm the expected tests ran and the function in sample.py did not run.

This approach ensures the test discovery rules are correctly followed by pytest.

Common Mistakes - 4 Pitfalls

{'mistake': "Naming test files without the 'test_' prefix or '_test' suffix", 'why_bad': "pytest will not discover test files that do not follow naming conventions, so tests inside those files won't run.", 'correct_approach': "Always name test files starting with 'test_' or ending with '_test.py' to ensure pytest discovers them."}

{'mistake': "Naming test functions without the 'test_' prefix", 'why_bad': "pytest only recognizes functions starting with 'test_' as test functions, so others are ignored.", 'correct_approach': "Prefix all test function names with 'test_' so pytest can find and run them."}

Running pytest with custom arguments that override discovery without understanding them

{'mistake': 'Hardcoding test paths or test names in automation scripts', 'why_bad': 'This reduces flexibility and makes tests brittle if file names or structure change.', 'correct_approach': "Use pytest's discovery features and relative paths to keep tests maintainable."}

Bonus Challenge

Now add data-driven testing with 3 different inputs to one of the test functions using pytest parametrize

Show Hint