Running collections via CLI in Postman - Framework Patterns

Framework Mode - Running collections via CLI

Folder Structure

postman-project/
├── collections/
│   └── my-collection.json          # Postman collection JSON file
├── environments/
│   └── dev-environment.json        # Environment variables for dev
├── reports/
│   └── test-report.html            # Generated test reports
├── scripts/
│   └── pre-request-scripts.js      # Optional helper scripts
├── newman-config.json              # Newman CLI configuration file
└── package.json                    # Node.js project config (if using npm)
  

Test Framework Layers

• Collections: JSON files defining API requests and tests.
• Environments: JSON files holding environment-specific variables.
• Scripts: Optional JavaScript files for pre-request or test scripts.
• Configuration: Newman CLI config file to define run options.
• Reports: Output folder for test run reports (HTML, JSON, etc.).

Configuration Patterns

Use separate environment JSON files for different setups (dev, staging, prod). Pass these to Newman CLI with --environment option.

Use a newman-config.json file or npm scripts to store common CLI options like reporters, iteration count, or timeout.

Store sensitive data in environment files and avoid committing secrets to version control.

Example CLI command:

newman run collections/my-collection.json --environment environments/dev-environment.json --reporters cli,html --reporter-html-export reports/test-report.html

Test Reporting and CI/CD Integration

Newman supports multiple reporters: CLI, HTML, JSON, JUnit XML.

Generate HTML or JUnit reports for easy viewing and integration with CI tools like Jenkins, GitHub Actions, or GitLab CI.

Example: Use --reporters cli,junit and --reporter-junit-export reports/results.xml to produce JUnit XML for CI pipelines.

Automate collection runs in CI by adding Newman commands in pipeline scripts.

Best Practices

• Keep collections and environments in separate folders for clarity.
• Use environment variables to avoid hardcoding URLs or credentials.
• Use Newman CLI options or config files to standardize test runs.
• Generate readable reports and integrate them into CI pipelines.
• Keep sensitive data out of version control by using environment files and .gitignore.

Self Check

Where would you add a new environment file for testing the staging server in this folder structure?

Key Result

Organize Postman collections, environments, and reports clearly; run tests via Newman CLI with config and integrate reports into CI.

Practice

1. What is the primary command line tool used to run Postman collections via CLI?

easy

A. npm

B. newman

C. curl

D. postman-cli

Solution

1. Step 1: Identify the official CLI tool for Postman collections

   Newman is the official command line tool designed to run Postman collections.

2. Step 2: Differentiate from other tools

   npm is a package manager, curl is a general HTTP client, and postman-cli is not an official tool.

3. Final Answer:

   newman -> Option B

4. Quick Check:

   Postman CLI tool = newman [OK]

Hint: Remember: Newman runs Postman collections in CLI [OK]

Common Mistakes:

• Confusing npm with the CLI runner
• Using curl as a Postman collection runner
• Assuming postman-cli is official

2. Which of the following is the correct syntax to run a Postman collection file named test_collection.json using Newman?

easy

A. newman execute test_collection.json

B. newman start test_collection.json

C. newman run test_collection.json

D. newman launch test_collection.json

Solution

1. Step 1: Recall the Newman command to run collections

   The correct command is newman run <collection-file>.

2. Step 2: Verify the options

   Options like execute, start, or launch are not valid Newman commands.

3. Final Answer:

   newman run test_collection.json -> Option C

4. Quick Check:

   Command syntax = newman run [OK]

Hint: Use 'newman run' to execute collections [OK]

Common Mistakes:

• Using 'execute' instead of 'run'
• Trying 'start' or 'launch' commands
• Omitting the 'run' keyword

3. Given the command newman run my_collection.json -e my_environment.json --delay-request 1000, what effect does the --delay-request 1000 option have?

medium

A. Limits the collection run to 1000 requests total

B. Sets the timeout for each request to 1000 milliseconds

C. Runs the collection 1000 times

D. Delays each request by 1000 milliseconds between executions

Solution

1. Step 1: Understand the --delay-request option

   This option adds a delay between each request in the collection, measured in milliseconds.

2. Step 2: Interpret the value 1000

   1000 milliseconds equals 1 second delay between requests.

3. Final Answer:

   Delays each request by 1000 milliseconds between executions -> Option D

4. Quick Check:

   --delay-request = wait time between requests [OK]

Hint: Delay means wait time between requests in ms [OK]

Common Mistakes:

• Confusing delay with timeout
• Thinking it repeats the collection
• Assuming it limits total requests

4. You run the command newman run collection.json -e env.json --reporters cli,html but get an error saying Unknown reporter: html. What is the most likely cause?

medium

A. The HTML reporter is not installed or supported by Newman

B. The environment file is invalid

C. The collection file is missing

D. The CLI reporter cannot be combined with others

Solution

1. Step 1: Analyze the error message

   The error states 'Unknown reporter: html', meaning Newman does not recognize the 'html' reporter.

2. Step 2: Understand reporter support

   Newman supports some reporters by default; others require installation or are unsupported.

3. Final Answer:

   The HTML reporter is not installed or supported by Newman -> Option A

4. Quick Check:

   Unknown reporter means unsupported or missing plugin [OK]

Hint: Check reporter names and installation before running [OK]

Common Mistakes:

• Blaming environment or collection files
• Assuming CLI reporter conflicts
• Ignoring reporter installation requirements

5. You want to run a Postman collection multiple times with different data sets from a CSV file named data.csv. Which Newman command correctly achieves this?

hard

A. newman run collection.json -d data.csv

B. newman run collection.json --data data.csv

C. newman run collection.json -e data.csv

D. newman run collection.json --env data.csv

Solution

1. Step 1: Identify the option for data files in Newman

   The -d or --iteration-data option is used to specify a data file for iteration.

2. Step 2: Differentiate environment and data options

   -e or --env is for environment files, not data files.

3. Final Answer:

   newman run collection.json -d data.csv -> Option A

4. Quick Check:

   Use -d for data files in Newman [OK]

Hint: Use -d to specify data file for iterations [OK]

Common Mistakes:

• Using -e or --env for data files
• Using --data, an invalid flag
• Confusing environment and data files