Why CLI execution enables automation in Postman - Automation Benefits in Action

Run Postman collection using CLI and verify successful execution

Preconditions (3)

Step 1: Open terminal or command prompt

Step 2: Navigate to the folder containing the Postman collection JSON file

Step 3: Run the command: newman run <collection-file-name>.json

Step 4: Observe the CLI output for test execution results

✅ Expected Result: The CLI output shows all requests executed with pass/fail status and summary of test results

Automation Requirements - Newman CLI

Assertions Needed:

Verify CLI output contains 'iterations' count

Verify CLI output shows 'failed' count as 0

Verify CLI output summary includes 'total requests' and 'total tests'

Best Practices:

Use explicit command line arguments for collection and environment files

Capture CLI output programmatically for assertions

Use exit codes to determine success or failure

Integrate CLI execution in CI/CD pipelines for automation

Automated Solution

Postman

import subprocess
import json

# Define the collection file path
collection_file = 'sample_collection.json'

# Run Newman CLI command
result = subprocess.run(['newman', 'run', collection_file], capture_output=True, text=True)

# Print CLI output
print(result.stdout)

# Basic assertions on CLI output
assert 'iterations' in result.stdout, 'Iterations count missing in output'
assert 'failed: 0' in result.stdout.lower(), 'There are failed tests'
assert 'total requests' in result.stdout.lower(), 'Total requests info missing'
assert result.returncode == 0, 'Newman CLI exited with error'

This script uses Python's subprocess module to run the Newman CLI command that executes the Postman collection.

We capture the CLI output as text to verify key information like iterations, failed tests, and total requests.

Assertions check that the output contains expected summary details and that the exit code is zero, indicating success.

This approach shows how CLI execution enables automation by allowing scripts to run tests and verify results without manual intervention.

Common Mistakes - 3 Pitfalls

Not capturing CLI output for verification

Ignoring exit codes from CLI execution

Hardcoding file paths without flexibility

Bonus Challenge

Now add data-driven testing by running the same collection with three different environment files

Show Hint

Practice

1. Why is running Postman tests via the CLI (Command Line Interface) useful for automation?

easy

A. It disables integration with other tools.

B. It requires a graphical interface to start tests.

C. It allows tests to run automatically without manual clicks.

D. It only works on Windows machines.

Solution

1. Step 1: Understand CLI automation benefits

   CLI lets you run tests by typing commands, so no manual clicking is needed.

2. Step 2: Compare with other options

   Options A, B, and C are false because CLI does not need a GUI, works on many OS, and supports integrations.

3. Final Answer:

   It allows tests to run automatically without manual clicks. -> Option C

4. Quick Check:

   CLI enables automation = It allows tests to run automatically without manual clicks. [OK]

Hint: CLI runs tests automatically, no manual steps needed [OK]

Common Mistakes:

• Thinking CLI needs a GUI
• Believing CLI works only on one OS
• Assuming CLI disables tool integration

2. Which of the following is the correct command to run a Postman collection using Newman CLI?

easy

A. postman execute collection.json

B. newman run collection.json

C. newman start collection.json

D. postman run collection.json

Solution

1. Step 1: Recall Newman CLI syntax

   The correct command to run a collection is 'newman run '.

2. Step 2: Check other options

   Options B, C, and D use incorrect commands or tools; only 'newman run' is valid.

3. Final Answer:

   newman run collection.json -> Option B

4. Quick Check:

   Newman run command = newman run collection.json [OK]

Hint: Use 'newman run' to execute collections via CLI [OK]

Common Mistakes:

• Using 'postman' instead of 'newman' in CLI
• Using 'start' instead of 'run'
• Confusing command syntax order

3. What will be the output when running this Newman CLI command?

newman run mycollection.json --reporters cli,json

medium

A. Runs tests and shows results in CLI and saves JSON report file.

B. Runs tests but only shows results in CLI, no JSON file created.

C. Syntax error because multiple reporters are not allowed.

D. Runs tests but ignores the JSON reporter option.

Solution

1. Step 1: Understand Newman reporters option

   Using '--reporters cli,json' runs tests and outputs results in CLI and JSON formats.

2. Step 2: Analyze each option

   Runs tests and shows results in CLI and saves JSON report file. correctly describes the output: runs tests and shows results in CLI and saves JSON report file. Options A, C, and D are incorrect because multiple reporters are allowed and JSON report is generated.

3. Final Answer:

   Runs tests and shows results in CLI and saves JSON report file. -> Option A

4. Quick Check:

   Multiple reporters output = Runs tests and shows results in CLI and saves JSON report file. [OK]

Hint: Multiple reporters output results in all specified formats [OK]

Common Mistakes:

• Thinking multiple reporters cause syntax errors
• Assuming JSON report is not saved
• Believing CLI ignores extra reporter options

4. You run this command but get an error:

newman run

What is the likely cause?

medium

A. The command should be 'newman execute'.

B. Newman CLI is not installed.

C. You need to add '--help' to run tests.

D. Missing collection file argument after 'run'.

Solution

1. Step 1: Check command syntax requirements

   'newman run' requires a collection file argument to specify which tests to run.

2. Step 2: Evaluate other options

   The command should be 'newman execute'. is wrong command ('execute' instead of 'run'). Newman CLI is not installed. might cause a different error like 'command not found'. You need to add '--help' to run tests. is unrelated to running tests.

3. Final Answer:

   Missing collection file argument after 'run'. -> Option D

4. Quick Check:

   Missing file argument = Missing collection file argument after 'run'. [OK]

Hint: Always provide collection file after 'newman run' [OK]

Common Mistakes:

• Forgetting to specify collection file
• Using wrong command 'execute'
• Thinking '--help' runs tests

5. How does running Postman tests via CLI help in continuous integration (CI) pipelines?

hard

A. It allows automated test runs triggered by code changes without manual intervention.

B. It requires manual approval before each test run.

C. It slows down the deployment process due to extra steps.

D. It only works if tests are run on a local machine.

Solution

1. Step 1: Understand CI pipeline automation

   CI pipelines run tests automatically when code changes are pushed, requiring no manual clicks.

2. Step 2: Evaluate options

   It allows automated test runs triggered by code changes without manual intervention. correctly states CLI enables this automation. Options B, C, and D contradict automation benefits or impose false limitations.

3. Final Answer:

   It allows automated test runs triggered by code changes without manual intervention. -> Option A

4. Quick Check:

   CLI enables CI automation = It allows automated test runs triggered by code changes without manual intervention. [OK]

Hint: CLI runs tests automatically in CI pipelines [OK]

Common Mistakes:

• Thinking manual approval is needed
• Believing CLI slows deployment
• Assuming tests must run locally