Test reporting in CI in Selenium Python - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is the purpose of test reporting in Continuous Integration (CI)?

Test reporting in CI helps track test results automatically after each code change. It shows if tests passed or failed, helping teams catch bugs early and maintain code quality.

beginner

Name a common format used for test reports in CI pipelines.

JUnit XML is a common format for test reports. It is easy to generate and many CI tools can read it to display test results clearly.

intermediate

How does Selenium integrate with CI tools for test reporting?

Selenium tests can be run in CI pipelines, and their results can be saved in report files like JUnit XML. CI tools then parse these files to show test summaries and details.

intermediate

What is a test report dashboard in CI?

A test report dashboard is a visual page in CI tools that shows test results over time. It helps teams see trends, failures, and test coverage at a glance.

beginner

Why is it important to have clear and accessible test reports in CI?

Clear test reports help developers quickly understand what failed and why. This speeds up fixing bugs and improves collaboration in the team.

Which file format is commonly used to store test results for CI tools?

A JUnit XML

B CSV

C TXT

D HTML

JUnit XML is widely supported by CI tools for parsing and displaying test results.

What does a failing test report in CI usually indicate?

A No tests were run

B Tests passed successfully

C CI server is offline

D Code changes caused a test to fail

A failing test report means some test did not pass after the latest code change.

How can Selenium test results be integrated into CI reports?

A By taking screenshots only

B By generating JUnit XML reports during test runs

C By manually writing test results in emails

D By running tests outside CI

Generating JUnit XML reports allows CI tools to automatically read and display Selenium test results.

What is a benefit of having test reports in CI dashboards?

A Run tests manually

B Hide test failures from the team

C Quickly see test trends and failures

D Ignore test results

Dashboards help teams monitor test health and react quickly to issues.

Why should test reports be easy to understand?

A To help developers fix bugs faster

B To confuse the team

C To delay bug fixes

D To avoid running tests

Clear reports speed up debugging and improve team collaboration.

Explain how test reporting works in a CI pipeline using Selenium tests.

Think about how test results move from Selenium to the CI tool.

Describe why test reporting is important for maintaining code quality in CI.

Consider the benefits of quick feedback on code changes.

Practice

1. What is the main purpose of test reporting in Continuous Integration (CI)?

easy

A. To provide quick feedback on test results automatically

B. To write test cases manually

C. To deploy the application to production

D. To create user documentation

Solution

1. Step 1: Understand CI test reporting role

   Test reporting in CI is designed to give fast feedback on whether tests pass or fail after code changes.

2. Step 2: Identify the correct purpose

   Among the options, only providing quick automatic feedback matches the main purpose of test reporting in CI.

3. Final Answer:

   To provide quick feedback on test results automatically -> Option A

4. Quick Check:

   Test reporting = quick feedback [OK]

Hint: Test reporting = fast feedback on tests [OK]

Common Mistakes:

• Confusing test reporting with deployment
• Thinking test reporting is about writing tests
• Mixing test reporting with documentation

2. Which pytest command option generates a test report in XML format suitable for CI tools?

easy

A. --html=report.html

B. --junitxml=report.xml

C. --json=report.json

D. --output=report.txt

Solution

1. Step 1: Recall pytest report options

   Pytest uses --junitxml to create XML reports compatible with many CI tools.

2. Step 2: Match option to XML report

   Among the options, only --junitxml=report.xml generates the correct XML format for CI.

3. Final Answer:

   --junitxml=report.xml -> Option B

4. Quick Check:

   Pytest XML report = --junitxml [OK]

Hint: Use --junitxml for CI XML reports in pytest [OK]

Common Mistakes:

• Using --html instead of --junitxml
• Confusing JSON or text output with XML
• Missing the file name after the option

3. Given this pytest command:
pytest tests/ --junitxml=results.xml
What will happen after the tests run?

medium

A. Tests will run but no report file will be created

B. Tests will fail due to syntax error in command

C. A file named results.xml will be created with test results in XML format

D. A HTML report named results.xml will be generated

Solution

1. Step 1: Understand the command effect

   The --junitxml=results.xml option tells pytest to save test results in XML format to the file named results.xml.

2. Step 2: Identify the output file type

   The file created is XML, not HTML, and the command syntax is correct, so tests run and report is saved.

3. Final Answer:

   A file named results.xml will be created with test results in XML format -> Option C

4. Quick Check:

   --junitxml creates XML report file [OK]

Hint: --junitxml creates XML report file after tests [OK]

Common Mistakes:

• Thinking no report is created
• Confusing XML with HTML report
• Assuming command syntax error

4. You added --junitxml=report.xml to your pytest command, but the CI tool shows no test report. What is the most likely cause?

medium

A. The tests did not run due to errors in test code

B. The pytest command syntax is incorrect

C. The CI tool does not support XML reports

D. The report.xml file is not saved in the CI workspace directory

Solution

1. Step 1: Check report file location

   CI tools need the report file in the workspace folder to read it. If saved elsewhere, the tool won't find it.

2. Step 2: Verify other causes

   The command syntax is correct, tests likely ran, and most CI tools support XML reports, so the missing file location is the main issue.

3. Final Answer:

   The report.xml file is not saved in the CI workspace directory -> Option D

4. Quick Check:

   Report file location matters in CI [OK]

Hint: Ensure report file is in CI workspace folder [OK]

Common Mistakes:

• Assuming command syntax error without checking
• Ignoring file save location
• Thinking CI tools don't support XML reports

5. In a CI pipeline, you want to fail the build if any Selenium tests fail and generate a detailed report. Which approach is best?

hard

A. Run pytest with --junitxml=report.xml and configure CI to fail on test failures using this report

B. Run tests without report and manually check logs after build

C. Use --html=report.html and ignore test failures in CI

D. Run tests locally and upload screenshots only

Solution

1. Step 1: Generate machine-readable test report

   Using --junitxml=report.xml creates a detailed XML report that CI tools can parse.

2. Step 2: Configure CI to fail build on test failures

   CI tools can read the XML report and mark the build as failed if tests fail, automating quality checks.

3. Final Answer:

   Run pytest with --junitxml=report.xml and configure CI to fail on test failures using this report -> Option A

4. Quick Check:

   Use XML report + CI fail on test fail [OK]

Hint: Use --junitxml and CI fail on test failure [OK]

Common Mistakes:

• Ignoring test failures in CI
• Not generating machine-readable reports
• Relying on manual log checks