Running collections via CLI in Postman - Build an Automation Script

Run Postman Collection using Newman CLI

Preconditions (2)

Step 1: Open the command line interface (CLI).

Step 2: Navigate to the directory containing 'sample_collection.json'.

Step 3: Run the command: newman run sample_collection.json

Step 4: Observe the CLI output for test run results.

✅ Expected Result: The CLI displays the execution summary showing the number of requests run, passed, and failed. All tests in the collection should pass without errors.

Automation Requirements - Newman CLI

Assertions Needed:

Verify that the CLI output contains 'iterations: 1'.

Verify that the CLI output contains 'requests: X' where X is the number of requests in the collection.

Verify that the CLI output contains 'tests: Y' where Y is the total number of tests run.

Verify that the CLI output contains 'failed: 0' indicating no test failures.

Best Practices:

Use explicit command line arguments for collection file path.

Capture CLI output programmatically for assertions.

Use JSON reporter for structured output parsing.

Handle errors gracefully if collection file is missing or invalid.

Automated Solution

Postman

import subprocess
import json

# Path to the Postman collection JSON file
collection_path = 'sample_collection.json'

# Run Newman CLI with JSON reporter for structured output
command = ['newman', 'run', collection_path, '--reporters', 'json']

try:
    result = subprocess.run(command, capture_output=True, text=True, check=True)
    # Parse the JSON output from Newman
    output_json = json.loads(result.stdout)

    # Extract run summary
    run_summary = output_json['run']['stats']

    # Assertions
    assert run_summary['iterations']['total'] == 1, f"Expected 1 iteration, got {run_summary['iterations']['total']}"
    assert run_summary['requests']['total'] >= 1, "Expected at least 1 request"
    assert run_summary['tests']['total'] >= 1, "Expected at least 1 test"
    assert run_summary['tests']['failed'] == 0, f"Expected 0 failed tests, got {run_summary['tests']['failed']}"

    print('Test run passed successfully.')

except subprocess.CalledProcessError as e:
    print(f'Newman CLI execution failed: {e}')
except json.JSONDecodeError:
    print('Failed to parse Newman JSON output.')
except AssertionError as ae:
    print(f'Assertion failed: {ae}')

This script automates running a Postman collection using Newman CLI.

We use Python's subprocess module to run the command newman run sample_collection.json --reporters json. The --reporters json option outputs the results in JSON format, which is easier to parse and assert.

We capture the CLI output and parse it as JSON. Then, we check the run summary statistics:

• There should be exactly 1 iteration.
• At least 1 request should have run.
• At least 1 test should have run.
• No tests should have failed.

If all assertions pass, the test run is successful. If any assertion fails, or if the CLI command fails, we print an error message.

This approach follows best practices by using structured output, explicit command arguments, and proper error handling.

Common Mistakes - 4 Pitfalls

{'mistake': 'Not using the JSON reporter and parsing plain text output.', 'why_bad': 'Plain text output is hard to parse reliably and can break if formatting changes.', 'correct_approach': "Use the '--reporters json' option to get structured output that can be parsed easily."}

Hardcoding the collection path without checking if the file exists.

Not capturing CLI output programmatically and relying on manual inspection.

Ignoring subprocess errors and not handling exceptions.

Bonus Challenge

Now add data-driven testing by running the same collection with three different environment files.

Show Hint