JUnit XML reporting for CI in PyTest - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

JUnit XML report generation with pytest

What is the output file generated by pytest when using the option --junitxml=report.xml?

PyTest

pytest --junitxml=report.xml

A No file is created; the option only prints XML to the console.

B A JSON file named <code>report.xml</code> is created with test results.

C A file named <code>report.xml</code> containing test results in JUnit XML format is created.

D A binary file named <code>report.xml</code> is created containing test logs.

💡 Hint

JUnit XML is a standard XML format for test reports used by CI tools.

The --junitxml=filename option tells pytest to write test results in JUnit XML format to the specified file. This file is XML, not JSON or binary.

❓ assertion

intermediate

Validating JUnit XML report content

Which assertion correctly checks that the JUnit XML report contains a test case with the name test_example?

PyTest

import xml.etree.ElementTree as ET

with open('report.xml') as f:
    tree = ET.parse(f)
    root = tree.getroot()

A assert root.find('testcase[@name="test_example"]') is None

B assert any(tc.attrib.get('name') == 'test_example' for tc in root.iter('testcase'))

C assert root.find('testcase').attrib['name'] == 'test_example'

D assert 'test_example' in root.text

💡 Hint

JUnit XML test cases are under elements with a 'name' attribute.

Option B correctly iterates over all elements and checks if any has the name 'test_example'. Option B checks for None which is opposite. Option B only checks the first testcase. Option B checks text content which is unreliable.

🔧 Debug

advanced

Fixing missing JUnit XML report in CI pipeline

A CI pipeline runs pytest with --junitxml=results.xml but the results.xml file is missing after the run. What is the most likely cause?

A The file was created but deleted by pytest after the run.

B The <code>--junitxml</code> option is deprecated and ignored by pytest.

C The test suite had no tests, so no report was generated.

D The pytest command ran in a different directory than expected, so the file was created elsewhere.

💡 Hint

Check the working directory of the pytest command in the CI job.

Pytest creates the JUnit XML file in the current working directory. If the CI job runs pytest in a different directory than expected, the file will be created there, not in the expected location. The option is not deprecated. Pytest creates the file even if no tests run. Pytest does not delete the file after creation.

❓ framework

advanced

Configuring pytest to always generate JUnit XML reports

How can you configure pytest to always generate a JUnit XML report named junit-report.xml without specifying --junitxml on the command line?

A Add <code>addopts = --junitxml=junit-report.xml</code> under the [pytest] section in <code>pytest.ini</code>.

B Set an environment variable <code>PYTEST_JUNITXML=junit-report.xml</code>.

C Create a <code>junit.xml</code> file manually before running pytest.

D Add <code>junitxml = junit-report.xml</code> under the [pytest] section in <code>pytest.ini</code>.

💡 Hint

pytest.ini can add default command line options.

The addopts setting in pytest.ini allows adding default command line options like --junitxml=filename. There is no junitxml config option. Creating a file manually or environment variable does not trigger report generation.

🧠 Conceptual

expert

Interpreting JUnit XML test suite attributes

In a JUnit XML report generated by pytest, what does the testsuite attribute failures represent?

A The number of test cases that failed due to assertion failures.

B The number of test cases skipped or marked as expected failures.

C The total number of tests executed including passed, failed, and skipped.

D The number of tests that passed successfully.

💡 Hint

Failures count tests that failed assertions.

The failures attribute counts tests that failed due to assertion failures. Skipped tests are counted separately. Total tests is in tests attribute. Passed tests are total minus failures, errors, and skipped.

Practice

1. What is the main purpose of generating a JUnit XML report in pytest for CI pipelines?

easy

A. To provide a standard format for test results that CI tools can read

B. To speed up test execution time

C. To automatically fix failing tests

D. To generate HTML reports for manual review

Solution

1. Step 1: Understand JUnit XML report role

   JUnit XML reports provide a standard format for test results that many CI tools can parse and display.

2. Step 2: Compare options with purpose

   Speeding tests, fixing tests, or generating HTML are not the main goals of JUnit XML reporting.

3. Final Answer:

   To provide a standard format for test results that CI tools can read -> Option A

4. Quick Check:

   JUnit XML = standard test result format [OK]

Hint: JUnit XML = standard test results for CI tools [OK]

Common Mistakes:

• Thinking JUnit XML speeds up tests
• Confusing XML report with HTML report
• Assuming it fixes tests automatically

2. Which pytest command correctly generates a JUnit XML report named results.xml?

easy

A. pytest --xml=results.xml

B. pytest --output=results.xml

C. pytest --report=results.xml

D. pytest --junitxml=results.xml

Solution

1. Step 1: Recall pytest option for JUnit XML

   The correct option to generate JUnit XML report is --junitxml=filename.

2. Step 2: Match option with command

   Only pytest --junitxml=results.xml uses --junitxml=results.xml, which is correct syntax.

3. Final Answer:

   pytest --junitxml=results.xml -> Option D

4. Quick Check:

   --junitxml option = correct flag [OK]

Hint: Use --junitxml=filename.xml to generate report [OK]

Common Mistakes:

• Using --xml instead of --junitxml
• Using --report or --output which are invalid
• Missing the equals sign '='

3. Given this pytest command:
pytest tests/ --junitxml=report.xml
What will happen after running it?

medium

A. Tests run but no report file is created

B. Tests run and a file named report.xml is created with test results in JUnit XML format

C. Tests do not run; only the report.xml file is created

D. Tests run and results are printed only to console, no file created

Solution

1. Step 1: Understand pytest command behavior

   The command runs tests in the tests/ folder and generates a JUnit XML report file named report.xml.

2. Step 2: Check output expectations

   Since --junitxml=report.xml is specified, the file will be created with test results in XML format.

3. Final Answer:

   Tests run and a file named report.xml is created with test results in JUnit XML format -> Option B

4. Quick Check:

   --junitxml creates XML report file [OK]

Hint: --junitxml creates XML report file after tests run [OK]

Common Mistakes:

• Thinking tests don't run when report option is used
• Expecting report file without running tests
• Assuming output only prints to console

4. You ran pytest --junitxml=results.xml but the file results.xml is empty. What is the most likely cause?

medium

A. No tests were collected or run

B. The --junitxml option is misspelled

C. The file path is invalid and pytest failed silently

D. JUnit XML reporting is not supported by pytest

Solution

1. Step 1: Analyze empty report cause

   An empty JUnit XML file usually means no tests were found or executed.

2. Step 2: Check other options

   The option --junitxml is correct, pytest supports it, and invalid paths usually cause errors, not silent empty files.

3. Final Answer:

   No tests were collected or run -> Option A

4. Quick Check:

   Empty XML = no tests run [OK]

Hint: Empty XML means no tests ran or were found [OK]

Common Mistakes:

• Assuming option is misspelled without checking
• Ignoring that no tests might be collected
• Thinking pytest doesn't support JUnit XML

5. You want your CI system to fail the build on the first pytest test failure using --failfast and --maxfail=1 and also save a JUnit XML report named ci-report.xml. Which command should you use?

hard

A. pytest --junitxml=ci-report.xml --maxfail=1 --exitfirst

B. pytest --junitxml=ci-report.xml --exitfirst --maxfail=0

C. pytest --junitxml=ci-report.xml --failfast --maxfail=1

D. pytest --junitxml=ci-report.xml --failfast --maxfail=0

Solution

1. Step 1: Understand options for failing fast

   To stop on first failure and fail the build, use --failfast and limit failures with --maxfail=1.

2. Step 2: Confirm JUnit XML report option

   --junitxml=ci-report.xml saves the report file as required.

3. Step 3: Evaluate options

   pytest --junitxml=ci-report.xml --failfast --maxfail=1 correctly combines --junitxml=ci-report.xml, --failfast, and --maxfail=1. Others have wrong or conflicting flags.

4. Final Answer:

   pytest --junitxml=ci-report.xml --failfast --maxfail=1 -> Option C

5. Quick Check:

   Fail fast + maxfail=1 + junitxml = pytest --junitxml=ci-report.xml --failfast --maxfail=1 [OK]

Hint: Use --failfast and --maxfail=1 with --junitxml for CI fail [OK]

Common Mistakes:

• Confusing --exitfirst with --failfast
• Setting maxfail=0 disables failure limit
• Omitting --failfast causes tests to run fully