Coverage in CI pipelines in PyTest - Framework Patterns

Framework Mode - Coverage in CI pipelines

Folder Structure

project-root/
├── tests/
│   ├── test_example.py
│   └── __init__.py
├── src/
│   └── app_code.py
├── .github/
│   └── workflows/
│       └── ci.yml
├── pytest.ini
├── requirements.txt
└── README.md
  

Test Framework Layers

• Test Layer: Located in tests/ folder, contains pytest test files like test_example.py.
• Application Code Layer: Source code under src/ folder.
• Configuration Layer: pytest.ini for pytest settings and coverage options.
• CI Pipeline Layer: GitHub Actions workflow file .github/workflows/ci.yml to run tests and collect coverage.
• Utilities: Optional helper scripts or fixtures inside tests/ or separate utils/ folder if needed.

Configuration Patterns

• pytest.ini: Configure pytest and coverage plugin settings, e.g., [pytest] section with addopts = --cov=src --cov-report=xml to generate coverage XML report.
• Environment Variables: Use environment variables in CI to control test behavior or credentials securely.
• CI Workflow: In ci.yml, install dependencies, run pytest with coverage, and upload coverage reports.
• Coverage Files: Store coverage data files like .coverage and reports in workspace for analysis.

Test Reporting and CI/CD Integration

• Run tests with coverage collection in CI pipeline (e.g., GitHub Actions).
• Generate coverage reports in XML or HTML format for easy viewing.
• Upload coverage reports to coverage services like Coveralls or Codecov for visualization and tracking over time.
• Fail CI build if coverage drops below a defined threshold using pytest-cov options.
• Integrate test results and coverage badges in project README for transparency.

Best Practices

1. Keep tests isolated: Each test should run independently to avoid false coverage results.
2. Use coverage thresholds: Enforce minimum coverage in CI to maintain code quality.
3. Generate multiple report formats: XML for CI tools, HTML for local inspection.
4. Secure secrets: Use encrypted environment variables for credentials in CI.
5. Automate coverage upload: Integrate with coverage services automatically after tests.

Self Check

Where in this folder structure would you add a new pytest fixture to share setup code across tests?

Key Result

Organize pytest tests and coverage config to run in CI pipelines with automated reporting and quality gates.