Custom markers in PyTest - Framework Patterns

Framework Mode - Custom markers

Folder Structure

tests/
├── test_login.py
├── test_checkout.py
├── test_profile.py
pytest.ini
conftest.py
utils/
    └── helpers.py

This is a typical pytest project structure. tests/ holds test files. pytest.ini configures pytest including custom markers. conftest.py contains fixtures and hooks. utils/ has helper functions.

Test Framework Layers

• Tests: Test functions in tests/ folder, using custom markers to group or select tests.
• Fixtures & Hooks: In conftest.py, provide setup/teardown and marker validation.
• Configuration: pytest.ini declares custom markers for pytest to recognize.
• Utilities: Helper functions in utils/ support test logic.

Configuration Patterns

Define custom markers in pytest.ini to avoid warnings and enable selection:

[pytest]
markers =
    smoke: mark test as smoke test
    regression: mark test as regression test
    slow: mark test as slow running

Use command line options to run tests by marker, e.g., pytest -m smoke.

Store environment variables or credentials securely outside tests, accessed via fixtures.

Test Reporting and CI/CD Integration

• Use pytest's built-in reporting with verbosity and markers to filter results.
• Integrate with CI/CD pipelines (GitHub Actions, Jenkins) to run tests with specific markers.
• Generate reports with plugins like pytest-html to show marker-based test groups.
• Fail builds if critical marker tests (e.g., smoke) fail.

Best Practices for Custom Markers

1. Declare all custom markers in pytest.ini to avoid warnings.
2. Use markers to categorize tests by purpose or speed (e.g., smoke, regression, slow).
3. Keep marker names clear and consistent.
4. Use markers in combination with fixtures for flexible test setup.
5. Leverage markers in CI to run targeted test suites efficiently.

Self Check

Where in this framework structure would you add a new custom marker called api to group API tests?

Key Result

Use pytest.ini to declare custom markers and apply them in tests for flexible grouping and selection.

Practice

1. What is the main purpose of using custom markers in pytest?

easy

A. To generate test data automatically

B. To change the Python interpreter version

C. To automatically fix test failures

D. To label tests for grouping and selective running

Solution

1. Step 1: Understand what custom markers do

   Custom markers in pytest are used to tag tests with labels for easier management.

2. Step 2: Identify the purpose of these labels

   These labels help run specific groups of tests selectively using the marker name.

3. Final Answer:

   To label tests for grouping and selective running -> Option D

4. Quick Check:

   Custom markers = test grouping [OK]

Hint: Markers tag tests to run groups easily [OK]

Common Mistakes:

• Thinking markers change Python version
• Assuming markers fix test errors automatically
• Confusing markers with test data generation

2. Which of the following is the correct syntax to mark a test with a custom marker named slow?

easy

A. @pytest.mark.slow\ndef test_example(): pass

B. @pytest.slow.mark\ndef test_example(): pass

C. @pytest.mark[slow]\ndef test_example(): pass

D. @pytest.mark('slow')\ndef test_example(): pass

Solution

1. Step 1: Recall pytest marker syntax

   Custom markers use dot notation: @pytest.mark.markername before the test function.

2. Step 2: Match the correct syntax

   @pytest.mark.slow is the correct way to apply the 'slow' marker.

3. Final Answer:

   @pytest.mark.slow\ndef test_example(): pass -> Option A

4. Quick Check:

   Marker syntax = @pytest.mark.name [OK]

Hint: Use dot notation: @pytest.mark.name [OK]

Common Mistakes:

• Using parentheses instead of dot notation
• Using square brackets instead of dot notation
• Placing marker after function name

3. Given the following test file, what will be the output when running pytest -m slow?

import pytest

@pytest.mark.slow
def test_a():
    assert True

def test_b():
    assert True

@pytest.mark.slow
def test_c():
    assert False

medium

A. Runs all tests; test_c fails

B. Runs only test_b; all pass

C. Runs test_a and test_c; test_c fails, test_a passes

D. Runs test_a and test_b; all pass

Solution

1. Step 1: Understand what pytest -m slow does

   This command runs only tests marked with the 'slow' marker.

2. Step 2: Identify marked tests in the code

   test_a and test_c have @pytest.mark.slow, so only these run.

3. Step 3: Check test results

   test_a passes (assert True), test_c fails (assert False).

4. Final Answer:

   Runs test_a and test_c; test_c fails, test_a passes -> Option C

5. Quick Check:

   pytest -m slow runs marked tests only [OK]

Hint: -m runs only tests with that marker [OK]

Common Mistakes:

• Assuming all tests run with -m option
• Ignoring test failures in marked tests
• Running unmarked tests by mistake

4. What is wrong with this pytest marker usage?

import pytest

@pytest.mark.slow()
def test_example():
    assert True

medium

A. Markers should not have parentheses unless parameters are used

B. Marker name must be uppercase

C. The test function must have parameters

D. pytest.mark cannot be used on test functions

Solution

1. Step 1: Check marker syntax

   Markers without parameters should not have parentheses; @pytest.mark.slow() is incorrect.

2. Step 2: Correct usage

   Use @pytest.mark.slow without parentheses for simple markers.

3. Final Answer:

   Markers should not have parentheses unless parameters are used -> Option A

4. Quick Check:

   Use parentheses only with marker args [OK]

Hint: No parentheses for simple markers [OK]

Common Mistakes:

• Adding empty parentheses to markers
• Using uppercase marker names incorrectly
• Thinking markers require function parameters

5. You want to run tests marked as database but exclude those also marked as slow. Which pytest command achieves this?

hard

A. pytest -m "database or slow"

B. pytest -m "database and not slow"

C. pytest -m "not database and slow"

D. pytest -m "database and slow"

Solution

1. Step 1: Understand marker expressions

   pytest supports boolean expressions with markers: and, or, not.

2. Step 2: Formulate the correct expression

   To run tests marked 'database' but exclude 'slow', use "database and not slow".

3. Final Answer:

   pytest -m "database and not slow" -> Option B

4. Quick Check:

   Use boolean logic in -m to include/exclude markers [OK]

Hint: Use boolean logic: -m "marker and not other" [OK]

Common Mistakes:

• Using 'or' instead of 'and not' to exclude
• Misplacing 'not' before the first marker
• Combining markers without quotes