Reporter options (CLI, HTML, JUnit) in Postman - Test Execution Trace

Test Overview

This test runs a Postman collection using Newman CLI and verifies that the test results are correctly reported in CLI, HTML, and JUnit formats.

Test Code - unittest

Postman

import subprocess
import os
import unittest

class TestNewmanReporters(unittest.TestCase):
    def setUp(self):
        self.collection = 'sample_collection.json'
        self.environment = 'sample_environment.json'
        self.report_dir = 'reports'
        os.makedirs(self.report_dir, exist_ok=True)

    def test_newman_cli_report(self):
        # Run Newman with CLI reporter
        result = subprocess.run([
            'newman', 'run', self.collection,
            '-e', self.environment
        ], capture_output=True, text=True)
        self.assertIn('iterations', result.stdout)
        self.assertEqual(result.returncode, 0)

    def test_newman_html_report(self):
        # Run Newman with HTML reporter
        html_report = os.path.join(self.report_dir, 'report.html')
        result = subprocess.run([
            'newman', 'run', self.collection,
            '-e', self.environment,
            '-r', 'html',
            '--reporter-html-export', html_report
        ], capture_output=True, text=True)
        self.assertTrue(os.path.exists(html_report))
        self.assertEqual(result.returncode, 0)

    def test_newman_junit_report(self):
        # Run Newman with JUnit reporter
        junit_report = os.path.join(self.report_dir, 'report.xml')
        result = subprocess.run([
            'newman', 'run', self.collection,
            '-e', self.environment,
            '-r', 'junit',
            '--reporter-junit-export', junit_report
        ], capture_output=True, text=True)
        self.assertTrue(os.path.exists(junit_report))
        self.assertEqual(result.returncode, 0)

if __name__ == '__main__':
    unittest.main()

Execution Trace - 4 Steps

Step	Action	System State	Assertion	Result
1	Test starts - Setup directories and files for reports	Reports directory created if not existing	—	PASS
2	Run Newman CLI reporter with collection and environment	Newman runs in terminal, outputs CLI summary	Check output contains 'iterations' and return code is 0	PASS
3	Run Newman with HTML reporter and export report	HTML report file generated in reports directory	Verify HTML report file exists and return code is 0	PASS
4	Run Newman with JUnit reporter and export report	JUnit XML report file generated in reports directory	Verify JUnit report file exists and return code is 0	PASS

Failure Scenario

Failing Condition: Newman command fails or report file is not generated

Execution Trace Quiz - 3 Questions

What does the CLI reporter output verify in the test?

A That the terminal output contains test summary like 'iterations'

B That an HTML file is created

C That a JUnit XML file is created

D That the collection file exists

Key Result

Always verify both the command execution success and the existence of generated report files to ensure reporter options work correctly.

Practice

1. Which reporter option in Postman is best for quickly viewing test results directly in the terminal?

easy

A. JSON reporter

B. HTML reporter

C. CLI reporter

D. JUnit reporter

Solution

1. Step 1: Understand reporter types

   CLI reporter shows results in the terminal quickly, HTML creates detailed reports, and JUnit is for integration.

2. Step 2: Match reporter to quick terminal output

   CLI reporter is designed for quick terminal output, so it fits the requirement.

3. Final Answer:

   CLI reporter -> Option C

4. Quick Check:

   Quick terminal output = CLI reporter [OK]

Hint: CLI means command line, perfect for terminal output [OK]

Common Mistakes:

• Confusing HTML reporter as quick terminal output
• Choosing JUnit reporter for terminal display
• Thinking JSON reporter is default quick output

2. Which of the following is the correct syntax to run a Postman collection with an HTML reporter using the CLI?

easy

A. newman run collection.json --reporter html

B. newman run collection.json --reporter-html

C. newman run collection.json -r html

D. newman run collection.json --reporters=html

Solution

1. Step 1: Recall Postman CLI reporter syntax

   The correct syntax uses '--reporter' followed by the reporter name without extra symbols.

2. Step 2: Check each option

   newman run collection.json --reporter html uses '--reporter html' which is correct. newman run collection.json --reporter-html and D have incorrect flags, and C uses shorthand '-r' which is invalid.

3. Final Answer:

   newman run collection.json --reporter html -> Option A

4. Quick Check:

   Correct flag is '--reporter html' [OK]

Hint: Use '--reporter html' exactly for HTML reports in CLI [OK]

Common Mistakes:

• Using '--reporter-html' instead of '--reporter html'
• Using shorthand '-r' which is not supported
• Adding '=' sign in '--reporters=html'

3. What will be the output when running this command?

newman run test_collection.json --reporter junit

Choose the best description.

medium

A. Displays test results in the terminal in JUnit style

B. Generates a JUnit XML report file for integration with CI tools

C. Creates an HTML report and opens it in the browser

D. Runs tests silently without any output

Solution

1. Step 1: Understand JUnit reporter purpose

   JUnit reporter generates XML files compatible with CI/CD tools for test result integration.

2. Step 2: Match command output

   The command uses '--reporter junit' which creates the XML report file, not terminal display or HTML report.

3. Final Answer:

   Generates a JUnit XML report file for integration with CI tools -> Option B

4. Quick Check:

   JUnit reporter = XML report file [OK]

Hint: JUnit reporter creates XML files for CI tools, not terminal output [OK]

Common Mistakes:

• Thinking JUnit reporter shows output in terminal
• Confusing JUnit with HTML reporter
• Assuming no output is generated

4. You run this command but get an error: newman run collection.json --reporter html junit. What is the likely cause?

medium

A. Missing comma between reporter names

B. Collection file name is invalid

C. HTML reporter is not supported in Newman

D. Incorrect flag usage for multiple reporters

Solution

1. Step 1: Check syntax for multiple reporters

   Newman requires multiple reporters to be specified with the '--reporters' flag and comma-separated list.

2. Step 2: Analyze given command

   The command uses '--reporter' once and lists reporters without commas, which is invalid syntax.

3. Final Answer:

   Incorrect flag usage for multiple reporters -> Option D

4. Quick Check:

   Use '--reporters html,junit' for multiple reporters [OK]

Hint: Use '--reporters html,junit' for multiple reporters [OK]

Common Mistakes:

• Using '--reporter' instead of '--reporters' for multiple
• Not separating reporter names with commas
• Assuming HTML reporter is unsupported

5. You want to run a Postman collection and generate both a detailed HTML report and a JUnit XML report for CI integration. Which command correctly achieves this?

hard

A. newman run collection.json --reporters html,junit

B. newman run collection.json --reporter html --reporter junit

C. newman run collection.json --reporters html;junit

D. newman run collection.json --reporters html junit

Solution

1. Step 1: Understand multiple reporter syntax

   Newman uses '--reporters' flag with comma-separated reporter names to run multiple reporters.

2. Step 2: Evaluate each option

   newman run collection.json --reporters html,junit uses correct syntax with '--reporters html,junit'. newman run collection.json --reporter html --reporter junit uses '--reporter' twice which is invalid. Options A and D use incorrect separators.

3. Final Answer:

   newman run collection.json --reporters html,junit -> Option A

4. Quick Check:

   Multiple reporters = '--reporters html,junit' [OK]

Hint: Use '--reporters' with comma-separated names for multiple reports [OK]

Common Mistakes:

• Using multiple '--reporter' flags instead of one '--reporters'
• Separating reporters with semicolon or space instead of comma
• Assuming only one reporter can be used