@pytest.mark.xfail for expected failures - Framework Patterns

Framework Mode - @pytest.mark.xfail for expected failures

Folder Structure

tests/
├── test_login.py
├── test_checkout.py
├── test_profile.py
conftest.py
pytest.ini
requirements.txt
utils/
├── helpers.py
├── data_provider.py

This is a typical Pytest project structure. All test files are inside tests/. Shared helpers and data are in utils/. Configuration is in pytest.ini.

Test Framework Layers

• Tests: Test functions or classes inside tests/ folder. Use @pytest.mark.xfail to mark tests expected to fail.
• Fixtures: Setup and teardown code in conftest.py to share test resources.
• Utilities: Helper functions and data providers in utils/ to keep tests clean.
• Configuration: pytest.ini to set options like markers, test paths, and logging.

Configuration Patterns

• pytest.ini: Declare custom markers like xfail to avoid warnings.
• Environment Variables: Use environment variables or pytest command options to select browsers or environments.
• Credentials: Store sensitive data outside code, e.g., in environment variables or encrypted files, accessed via fixtures.
• Example pytest.ini snippet:

  [pytest]
  markers =
      xfail: mark test as expected to fail
  

Test Reporting and CI/CD Integration

• Use pytest built-in reports for pass/fail and xfail outcomes.
• Integrate with CI tools (GitHub Actions, Jenkins) to run tests on code changes.
• Configure CI to fail build only if unexpected failures occur (xfail tests do not fail the build).
• Use plugins like pytest-html for detailed HTML reports showing which tests were expected to fail.

Best Practices

• Use @pytest.mark.xfail only for known bugs or incomplete features.
• Always add a reason to xfail marker for clarity, e.g., @pytest.mark.xfail(reason="bug #123").
• Do not ignore unexpected passes; monitor xfail tests that start passing to update or remove the marker.
• Keep tests independent; xfail should not hide failures in other tests.
• Use fixtures to manage test setup cleanly, so xfail tests run in the same environment.

Self Check

Where in this folder structure would you add a new test that is expected to fail because of a known bug?

Key Result

Use @pytest.mark.xfail to mark tests expected to fail, keeping test results clear and manageable.