Custom markers in PyTest - Build an Automation Script

Automate test execution with a custom pytest marker

Preconditions (3)

Step 1: Run pytest with the option to only execute tests marked with 'slow'

Step 2: Verify that only the test(s) with the 'slow' marker are executed

Step 3: Verify that tests without the 'slow' marker are skipped

Step 4: Check the test report to confirm the number of tests run matches the number of 'slow' marked tests

✅ Expected Result: Only tests marked with the 'slow' marker run and pass or fail as expected; other tests are skipped; test report shows correct test count

Automation Requirements - pytest

Assertions Needed:

Assert that the test output includes only tests with the 'slow' marker

Assert that tests without the 'slow' marker are not run

Assert that the test report summary matches the expected number of tests run

Best Practices:

Use pytest's @pytest.mark decorator to mark tests

Use pytest command line option '-m' to select tests by marker

Use pytest's capsys or caplog fixtures to capture output if needed

Avoid hardcoding test names; rely on marker filtering

Include marker registration in pytest.ini to avoid warnings

Automated Solution

PyTest

import pytest
import subprocess
import sys

# test_sample.py content
# Save this content in the same directory before running automation
#
# import pytest
#
# @pytest.mark.slow
# def test_slow_example():
#     assert True
#
# def test_fast_example():
#     assert True


def test_run_only_slow_marker():
    # Run pytest with -m slow to run only tests marked as slow
    result = subprocess.run(
        [sys.executable, '-m', 'pytest', '-v', '-m', 'slow', 'test_sample.py'],
        capture_output=True,
        text=True
    )

    output = result.stdout

    # Assert the command ran successfully
    assert result.returncode == 0, f"Pytest failed with output:\n{output}"

    # Check that the output contains the slow test name
    assert 'test_slow_example' in output, "Slow test did not run"

    # Check that the fast test did not run
    assert 'test_fast_example' not in output, "Fast test should not run when filtering by slow marker"

    # Check the test summary line for correct count
    # Example summary line: '1 passed in 0.03s'
    import re
    match = re.search(r'(\d+) passed', output)
    assert match is not None, "No tests passed"
    passed_count = int(match.group(1))
    assert passed_count == 1, f"Expected 1 test to run, but {passed_count} ran"

This automation script runs pytest as a subprocess with the -m slow option to select only tests marked with the slow marker.

It captures the output and checks:

• The pytest command succeeded (return code 0).
• The output contains the slow test name test_slow_example, confirming it ran.
• The output does not contain the fast test name test_fast_example, confirming it was skipped.
• The test summary shows exactly one test passed, matching the expected count.

This approach uses subprocess to run pytest as a real user would, ensuring marker filtering works as expected.

Before running this automation, ensure test_sample.py exists with the given test functions and the slow marker is registered in pytest.ini to avoid warnings.

Common Mistakes - 4 Pitfalls

Not registering the custom marker in pytest.ini

Using hardcoded test names in assertions

Running pytest without the -m option to filter markers

Not capturing output to verify which tests ran

Bonus Challenge

Now add data-driven testing with 3 different inputs to the slow test using pytest.mark.parametrize

Show Hint

Practice

1. What is the main purpose of using custom markers in pytest?

easy

A. To generate test data automatically

B. To change the Python interpreter version

C. To automatically fix test failures

D. To label tests for grouping and selective running

Solution

1. Step 1: Understand what custom markers do

   Custom markers in pytest are used to tag tests with labels for easier management.

2. Step 2: Identify the purpose of these labels

   These labels help run specific groups of tests selectively using the marker name.

3. Final Answer:

   To label tests for grouping and selective running -> Option D

4. Quick Check:

   Custom markers = test grouping [OK]

Hint: Markers tag tests to run groups easily [OK]

Common Mistakes:

• Thinking markers change Python version
• Assuming markers fix test errors automatically
• Confusing markers with test data generation

2. Which of the following is the correct syntax to mark a test with a custom marker named slow?

easy

A. @pytest.mark.slow\ndef test_example(): pass

B. @pytest.slow.mark\ndef test_example(): pass

C. @pytest.mark[slow]\ndef test_example(): pass

D. @pytest.mark('slow')\ndef test_example(): pass

Solution

1. Step 1: Recall pytest marker syntax

   Custom markers use dot notation: @pytest.mark.markername before the test function.

2. Step 2: Match the correct syntax

   @pytest.mark.slow is the correct way to apply the 'slow' marker.

3. Final Answer:

   @pytest.mark.slow\ndef test_example(): pass -> Option A

4. Quick Check:

   Marker syntax = @pytest.mark.name [OK]

Hint: Use dot notation: @pytest.mark.name [OK]

Common Mistakes:

• Using parentheses instead of dot notation
• Using square brackets instead of dot notation
• Placing marker after function name

3. Given the following test file, what will be the output when running pytest -m slow?

import pytest

@pytest.mark.slow
def test_a():
    assert True

def test_b():
    assert True

@pytest.mark.slow
def test_c():
    assert False

medium

A. Runs all tests; test_c fails

B. Runs only test_b; all pass

C. Runs test_a and test_c; test_c fails, test_a passes

D. Runs test_a and test_b; all pass

Solution

1. Step 1: Understand what pytest -m slow does

   This command runs only tests marked with the 'slow' marker.

2. Step 2: Identify marked tests in the code

   test_a and test_c have @pytest.mark.slow, so only these run.

3. Step 3: Check test results

   test_a passes (assert True), test_c fails (assert False).

4. Final Answer:

   Runs test_a and test_c; test_c fails, test_a passes -> Option C

5. Quick Check:

   pytest -m slow runs marked tests only [OK]

Hint: -m runs only tests with that marker [OK]

Common Mistakes:

• Assuming all tests run with -m option
• Ignoring test failures in marked tests
• Running unmarked tests by mistake

4. What is wrong with this pytest marker usage?

import pytest

@pytest.mark.slow()
def test_example():
    assert True

medium

A. Markers should not have parentheses unless parameters are used

B. Marker name must be uppercase

C. The test function must have parameters

D. pytest.mark cannot be used on test functions

Solution

1. Step 1: Check marker syntax

   Markers without parameters should not have parentheses; @pytest.mark.slow() is incorrect.

2. Step 2: Correct usage

   Use @pytest.mark.slow without parentheses for simple markers.

3. Final Answer:

   Markers should not have parentheses unless parameters are used -> Option A

4. Quick Check:

   Use parentheses only with marker args [OK]

Hint: No parentheses for simple markers [OK]

Common Mistakes:

• Adding empty parentheses to markers
• Using uppercase marker names incorrectly
• Thinking markers require function parameters

5. You want to run tests marked as database but exclude those also marked as slow. Which pytest command achieves this?

hard

A. pytest -m "database or slow"

B. pytest -m "database and not slow"

C. pytest -m "not database and slow"

D. pytest -m "database and slow"

Solution

1. Step 1: Understand marker expressions

   pytest supports boolean expressions with markers: and, or, not.

2. Step 2: Formulate the correct expression

   To run tests marked 'database' but exclude 'slow', use "database and not slow".

3. Final Answer:

   pytest -m "database and not slow" -> Option B

4. Quick Check:

   Use boolean logic in -m to include/exclude markers [OK]

Hint: Use boolean logic: -m "marker and not other" [OK]

Common Mistakes:

• Using 'or' instead of 'and not' to exclude
• Misplacing 'not' before the first marker
• Combining markers without quotes