Verify pytest configuration standardizes test behavior

Preconditions (2)

Step 1: Create two test functions: one marked with @pytest.mark.smoke and one without any marker

Step 2: Run pytest with the option '-m smoke' to select only tests with the smoke marker

Step 3: Observe which tests run

Step 4: Run pytest without any marker option

Step 5: Observe which tests run

✅ Expected Result: When running pytest with '-m smoke', only the test marked with @pytest.mark.smoke runs. When running pytest without '-m smoke', all tests run. This shows configuration standardizes which tests run.

Automation Requirements - pytest

Assertions Needed:

Assert that only the smoke test runs when pytest is run with '-m smoke'

Assert that both tests run when pytest is run without '-m smoke'

Best Practices:

Use pytest markers properly

Use pytest's capsys or pytester plugin to capture test run output

Keep tests independent and clear

Use pytest.ini to define markers

Automated Solution

PyTest

import subprocess
import sys
import pytest

# Content of test_sample.py
TEST_CODE = '''
import pytest

@pytest.mark.smoke
 def test_smoke():
    assert True

def test_regular():
    assert True
'''

# Content of pytest.ini
PYTEST_INI = '''
[pytest]
markers =
    smoke: mark a test as smoke test
'''

def test_pytest_marker_behavior(tmp_path):
    # Write test file
    test_file = tmp_path / "test_sample.py"
    test_file.write_text(TEST_CODE)

    # Write pytest.ini
    ini_file = tmp_path / "pytest.ini"
    ini_file.write_text(PYTEST_INI)

    # Run pytest with -m smoke
    result_smoke = subprocess.run(
        [sys.executable, '-m', 'pytest', str(test_file), '-m', 'smoke', '-q', '--tb=no'],
        capture_output=True,
        text=True,
        cwd=tmp_path
    )

    # Run pytest without marker
    result_all = subprocess.run(
        [sys.executable, '-m', 'pytest', str(test_file), '-q', '--tb=no'],
        capture_output=True,
        text=True,
        cwd=tmp_path
    )

    # Check output for smoke run: only test_smoke should run
    assert 'test_smoke' in result_smoke.stdout
    assert 'test_regular' not in result_smoke.stdout

    # Check output for all tests run: both tests should run
    assert 'test_smoke' in result_all.stdout
    assert 'test_regular' in result_all.stdout

This test creates a temporary test file with two tests: one marked with @pytest.mark.smoke and one regular test.

It also creates a pytest.ini file defining the smoke marker to avoid warnings.

Then it runs pytest twice using subprocess: once with -m smoke to run only smoke tests, and once without any marker to run all tests.

The assertions check that when running with -m smoke, only the smoke test runs, and when running without it, both tests run.

This shows how configuration (markers and pytest.ini) standardizes which tests run, making test behavior predictable and manageable.

Common Mistakes - 3 Pitfalls

Not defining custom markers in pytest.ini

Using print statements instead of assertions to verify test selection

Running pytest commands without capturing output in automation

Bonus Challenge

Now add data-driven testing with 3 different inputs to the smoke test and verify all run with '-m smoke'

Show Hint