Test reporting in CI in Selenium Python - Framework Patterns

Framework Mode - Test reporting in CI

Folder Structure

selenium-python-project/
├── tests/
│   ├── test_login.py
│   ├── test_checkout.py
│   └── __init__.py
├── pages/
│   ├── login_page.py
│   ├── checkout_page.py
│   └── __init__.py
├── utils/
│   ├── driver_factory.py
│   ├── logger.py
│   └── __init__.py
├── reports/
│   └── (generated test reports here)
├── config/
│   ├── config.yaml
│   └── __init__.py
├── conftest.py
├── requirements.txt
└── pytest.ini

Test Framework Layers

• Driver Layer: Handles browser setup and teardown (e.g., driver_factory.py).
• Page Objects: Encapsulate page elements and actions (e.g., login_page.py).
• Tests: Test cases using pytest that call page objects (e.g., test_login.py).
• Utilities: Helpers like logging, screenshot capture, and report generation.
• Configuration: Environment settings, credentials, and browser options (e.g., config.yaml).

Configuration Patterns

• Environment Config: Use config/config.yaml to store URLs, credentials, and browser types for different environments (dev, staging, prod).
• pytest.ini: Configure pytest options like markers and test paths.
• Command Line Overrides: Use pytest command line options to select environment or browser dynamically.
• Secrets Management: Store sensitive data outside the repo or use environment variables.

Test Reporting and CI/CD Integration

• Test Reports: Use pytest plugins like pytest-html or pytest-allure to generate readable HTML or XML reports saved in reports/ folder.
• CI Integration: Configure CI tools (e.g., GitHub Actions, Jenkins) to run tests on code push and collect reports as build artifacts.
• Fail Fast and Notifications: Configure CI to fail the build on test failures and send notifications (email, Slack).
• Report Archiving: Keep historical test reports for trend analysis and debugging.

Best Practices for Test Reporting in CI

1. Clear and Readable Reports: Use HTML or Allure reports with screenshots on failure for easy debugging.
2. Automate Report Generation: Integrate report creation in test runs automatically.
3. Store Reports as Artifacts: Save reports in CI to access after runs.
4. Fail Builds on Test Failures: Prevent broken code from merging by failing CI on test errors.
5. Use Tags and Filters: Organize tests with markers to run subsets and generate focused reports.

Self Check

Where in this folder structure would you add a new test report plugin configuration to generate HTML reports automatically?

Key Result

Organize Selenium Python tests with clear layers and generate automated HTML reports integrated into CI pipelines.