Test result publishing in PyTest - Framework Patterns

Framework Mode - Test result publishing

Folder Structure

project-root/
├── tests/
│   ├── test_example.py
│   └── conftest.py
├── reports/
│   └── latest_report.html
├── pytest.ini
├── requirements.txt
└── ci/
    └── pipeline.yml
    

Test Framework Layers

• Tests Layer: Contains test cases using pytest functions or classes inside tests/.
• Fixtures & Setup: conftest.py holds reusable setup code and fixtures for tests.
• Reporting Layer: Generates test result files (HTML, XML) saved in reports/.
• Configuration Layer: pytest.ini configures pytest options like markers and report formats.
• CI/CD Layer: Pipeline scripts in ci/ automate test runs and publish reports.

Configuration Patterns

Use pytest.ini to set default options, for example:

[pytest]
addopts = --html=reports/latest_report.html --self-contained-html --junitxml=reports/results.xml
markers =
    smoke: Quick smoke tests
    regression: Full regression suite
    

Manage environments and credentials via environment variables or separate config files loaded in conftest.py.

Example: Use os.environ in fixtures to select test environment or browser.

Test Reporting and CI/CD Integration

• Generate HTML reports using pytest-html plugin for easy visual results.
• Generate XML reports (JUnit format) for CI tools like Jenkins, GitHub Actions.
• Store reports in reports/ folder for easy access and archiving.
• CI pipeline (ci/pipeline.yml) runs tests, collects reports, and publishes them as build artifacts or uploads to dashboards.
• Example: GitHub Actions workflow triggers pytest, uploads reports/latest_report.html as an artifact.

Best Practices for Test Result Publishing

1. Consistent Report Location: Always save reports in a dedicated folder like reports/ for easy retrieval.
2. Use Standard Formats: Generate both human-readable (HTML) and machine-readable (XML) reports.
3. Integrate with CI/CD: Automate test runs and report publishing in your pipeline for fast feedback.
4. Keep Reports Self-Contained: Use plugins options like --self-contained-html to avoid missing resources.
5. Secure Sensitive Data: Avoid printing credentials or sensitive info in reports.

Self Check

Where in this folder structure would you add a new pytest plugin configuration to generate a JSON test report?

Key Result

Organize pytest tests, configs, and reports clearly; automate report generation and publishing in CI/CD pipelines.

Practice

1. What is the main purpose of publishing test results in pytest?

easy

A. To change the test code behavior

B. To speed up test execution time

C. To write more test cases automatically

D. To share test outcomes with team members or tools

Solution

1. Step 1: Understand test result publishing

   Publishing test results means sharing what happened during tests with others.

2. Step 2: Identify the purpose in pytest context

   In pytest, publishing results helps teams see which tests passed or failed.

3. Final Answer:

   To share test outcomes with team members or tools -> Option D

4. Quick Check:

   Test result publishing = sharing outcomes [OK]

Hint: Publishing means sharing test results with others [OK]

Common Mistakes:

• Thinking publishing speeds up tests
• Confusing publishing with writing tests
• Believing publishing changes test code

2. Which pytest command correctly saves test results to an XML file named results.xml?

easy

A. pytest --junitxml=results.xml

B. pytest --savexml=results.xml

C. pytest --output=results.xml

D. pytest --xmlfile=results.xml

Solution

1. Step 1: Recall pytest command for XML report

   The correct option uses --junitxml to save results in XML format.

2. Step 2: Match the command with the filename

   Using pytest --junitxml=results.xml saves the test report as results.xml.

3. Final Answer:

   pytest --junitxml=results.xml -> Option A

4. Quick Check:

   Use --junitxml to save XML report [OK]

Hint: Use --junitxml=filename.xml to save results [OK]

Common Mistakes:

• Using --savexml instead of --junitxml
• Using --output or --xmlfile which are invalid
• Forgetting the equals sign '='

3. Given this pytest command:

pytest --junitxml=report.xml

What will happen after running tests?

medium

A. Test results will be saved in a file named report.xml

B. Tests will not run due to syntax error

C. Tests will run but no results will be saved

D. Test results will be printed only on the console

Solution

1. Step 1: Understand the command option

   The option --junitxml=report.xml tells pytest to save results in XML format to the file report.xml.

2. Step 2: Predict the outcome after running tests

   Tests run normally and results are saved in the specified XML file.

3. Final Answer:

   Test results will be saved in a file named report.xml -> Option A

4. Quick Check:

   --junitxml saves results to XML file [OK]

Hint: Look for --junitxml to save results to XML file [OK]

Common Mistakes:

• Thinking results only print on console
• Assuming tests won't run with this option
• Confusing file saving with syntax errors

4. You run pytest --junitxml report.xml but no report.xml file is created. What is the likely problem?

medium

A. pytest does not support XML reports

B. The command is missing an equals sign between option and filename

C. The test files are empty so no report is generated

D. The filename must be results.xml exactly

Solution

1. Step 1: Check the command syntax

   The correct syntax requires an equals sign: --junitxml=report.xml.

2. Step 2: Understand effect of missing equals sign

   Without '=', pytest treats report.xml as a positional argument, so no file is created.

3. Final Answer:

   The command is missing an equals sign between option and filename -> Option B

4. Quick Check:

   Use '=' after --junitxml to save file [OK]

Hint: Remember to use '=' after --junitxml option [OK]

Common Mistakes:

• Omitting the equals sign in command
• Thinking empty tests prevent report creation
• Believing pytest can't create XML reports

5. You want to publish pytest results in XML and also ensure the report includes test durations. Which command should you use?

hard

A. pytest --xmlreport=report.xml --showdurations

B. pytest --junitxml=report.xml --duration=all

C. pytest --junitxml=report.xml --durations=0

D. pytest --junitxml report.xml --durations

Solution

1. Step 1: Identify correct option for XML report

   The option --junitxml=report.xml correctly saves results in XML format.

2. Step 2: Add option to show all test durations

   The option --durations=0 tells pytest to show durations for all tests.

3. Step 3: Verify combined command correctness

   Combining both options as pytest --junitxml=report.xml --durations=0 is valid and achieves the goal.

4. Final Answer:

   pytest --junitxml=report.xml --durations=0 -> Option C

5. Quick Check:

   Use --junitxml and --durations=0 together [OK]

Hint: Use --junitxml=filename and --durations=0 for full report [OK]

Common Mistakes:

• Using wrong option names like --xmlreport
• Missing equals sign in options
• Using incorrect values like --duration=all