Collection runner results in Postman - Build an Automation Script

Run a Postman collection and verify the results

Preconditions (3)

Step 1: Open the Postman app

Step 2: Select the collection to run

Step 3: Click the 'Run' button to open the Collection Runner

Step 4: In the Collection Runner, click 'Start Run' to execute all requests

Step 5: Wait for the collection run to complete

Step 6: Observe the results summary showing number of requests, passed tests, and failed tests

✅ Expected Result: The collection runner completes all requests, showing a summary with the total requests run, number of passed tests, and number of failed tests. Each request's test results are displayed correctly.

Automation Requirements - Newman (Postman CLI)

Assertions Needed:

Verify the total number of requests run matches the collection size

Verify all tests in each request pass

Verify the summary shows zero failed tests

Best Practices:

Use Newman CLI to run collections programmatically

Parse the JSON output from Newman to assert test results

Use explicit assertions on summary statistics

Handle asynchronous execution properly

Automated Solution

Postman

import { exec } from 'child_process';

const collectionPath = './my_collection.json';

exec(`newman run ${collectionPath} -r json`, (error, stdout, stderr) => {
  if (error) {
    console.error(`Execution error: ${error}`);
    process.exit(1);
  }

  try {
    // Newman outputs JSON report after the run
    const outputStart = stdout.indexOf('{');
    const outputJson = stdout.substring(outputStart);
    const result = JSON.parse(outputJson);

    const totalRequests = result.run.stats.requests.total;
    const totalTests = result.run.stats.tests.total;
    const failedTests = result.run.stats.tests.failed;

    console.assert(totalRequests === result.run.executions.length, `Expected total requests ${result.run.executions.length}, got ${totalRequests}`);
    console.assert(failedTests === 0, `Expected 0 failed tests, got ${failedTests}`);

    console.log('Collection run successful with all tests passing.');
  } catch (parseError) {
    console.error('Failed to parse Newman JSON output:', parseError);
    process.exit(1);
  }
});

This script uses Node.js child_process.exec to run Newman CLI with the -r json reporter, which outputs the collection run results in JSON format.

It captures the standard output, extracts the JSON part, and parses it to an object.

Assertions check that the total number of requests run matches the number of executions and that there are zero failed tests.

If assertions pass, it logs success; otherwise, it throws an error.

This approach automates running the Postman collection and verifying the collection runner results programmatically.

Common Mistakes - 3 Pitfalls

Parsing Newman output as plain text without extracting JSON

Not asserting the number of requests run

Ignoring failed tests count in assertions

Bonus Challenge

Now add data-driven testing by running the collection with three different environment files

Show Hint