Running collections via CLI in Postman - Test Execution Trace

Test Overview

This test runs a Postman collection using the CLI tool Newman. It verifies that the collection executes successfully and all tests inside pass.

Test Code - Python subprocess with Newman CLI

Postman

import subprocess

# Run Postman collection using Newman CLI
result = subprocess.run([
    'newman', 'run', 'sample_collection.json',
    '--reporters', 'cli',
    '--timeout-request', '5000'
], capture_output=True, text=True)

# Check if the run was successful by looking for 'run complete' in output
assert 'run complete' in result.stdout.lower(), 'Collection run did not complete successfully'

# Check that there are no failed tests reported
assert 'failed: 0' in result.stdout.lower(), 'Some tests failed in the collection run'

Execution Trace - 6 Steps

Step	Action	System State	Assertion	Result
1	Test starts	Test script is ready to run the Newman CLI command	-	PASS
2	Runs 'newman run sample_collection.json --reporters cli --timeout-request 5000' via subprocess	Newman CLI starts executing the Postman collection	-	PASS
3	Newman executes all requests and tests inside the collection	CLI output shows request results and test assertions	-	PASS
4	Test script checks if 'run complete' is in the CLI output	CLI output contains summary of the run	'run complete' found in output	PASS
5	Test script checks if 'failed: 0' is in the CLI output	CLI output shows zero failed tests	'failed: 0' found in output	PASS
6	Test ends with all assertions passed	Collection run successful with no test failures	-	PASS

Failure Scenario

Failing Condition: Newman CLI run does not complete or tests fail in the collection

Execution Trace Quiz - 3 Questions

What does the test check to confirm the collection ran successfully?

A Presence of 'run complete' in CLI output

B Presence of 'error' in CLI output

C Number of requests in the collection

D Size of the collection file

Key Result

Always verify both the completion status and the test results when running collections via CLI to ensure the entire suite executed successfully without failures.