Command-line options in PyTest - Framework Patterns

Framework Mode - Command-line options

Folder Structure

project-root/
├── tests/
│   ├── test_example.py
│   └── test_login.py
├── conftest.py
├── pytest.ini
├── requirements.txt
└── README.md
  

Test Framework Layers

• Tests: Located in tests/ folder, contains test files like test_example.py.
• Fixtures & Hooks: Defined in conftest.py for setup and teardown, shared resources.
• Configuration: pytest.ini file to set default command-line options and test discovery rules.
• Utilities: Helper functions or modules can be added as needed for reusable code.

Configuration Patterns

Use pytest.ini to define default command-line options and customize test runs:

[pytest]
addopts = -v --maxfail=2 --tb=short
markers =
    smoke: Quick smoke tests
    regression: Full regression tests
  

Command-line options examples:

• pytest -v: verbose output
• pytest -k "login and not slow": run tests matching expression
• pytest --maxfail=1: stop after first failure
• pytest --tb=short: shorter traceback on failure

Use conftest.py to add custom command-line options with pytest_addoption hook.

# conftest.py
import pytest

def pytest_addoption(parser):
    parser.addoption(
        "--env",
        action="store",
        default="dev",
        help="Environment to run tests against: dev, staging, prod"
    )

@pytest.fixture
def env(request):
    return request.config.getoption("--env")
  

Test Reporting and CI/CD Integration

• Use built-in pytest options like --junitxml=report.xml to generate XML reports for CI tools.
• Integrate with CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI) by running pytest with desired command-line options.
• Example GitHub Actions step:

- name: Run tests
  run: |
    pytest -v --junitxml=results.xml
  

This allows CI to parse test results and show pass/fail status.

Best Practices

1. Define common command-line options in pytest.ini to keep test runs consistent.
2. Use conftest.py to add custom options for flexibility (e.g., environment selection).
3. Keep command-line options simple and meaningful to avoid confusion.
4. Use markers and -k expressions to select specific tests easily.
5. Integrate reporting options with CI/CD for automated feedback.

Self Check

Where in this framework structure would you add a new custom command-line option to select a browser type for tests?

Key Result

Use pytest.ini for default options and conftest.py to add custom command-line options for flexible test runs.

Practice

1. What does the pytest command-line option -v do when running tests?

easy

A. Runs tests only once without reruns

B. Runs tests in quiet mode, suppressing output

C. Generates a test coverage report

D. Runs tests in verbose mode, showing each test name

Solution

1. Step 1: Understand the -v option in pytest

   The -v option stands for verbose mode, which means pytest will show detailed information about each test it runs.

2. Step 2: Compare with other options

   Options like quiet mode or coverage report are different flags, so -v specifically increases output detail.

3. Final Answer:

   Runs tests in verbose mode, showing each test name -> Option D

4. Quick Check:

   Verbose = detailed output [OK]

Hint: Verbose means more details shown during test run [OK]

Common Mistakes:

• Confusing verbose with quiet mode
• Thinking it runs tests only once
• Mixing it with coverage report option

2. Which of the following is the correct syntax to run pytest and stop after the first failure?

easy

A. pytest -x

B. pytest --stop-on-failure

C. pytest --failfast=false

D. pytest -f

Solution

1. Step 1: Recall pytest option to stop on first failure

   The correct option is -x, which tells pytest to stop after the first test failure.

2. Step 2: Check other options for correctness

   --stop-on-failure and --failfast=false are invalid, and -f is not a recognized pytest option.

3. Final Answer:

   pytest -x -> Option A

4. Quick Check:

   Stop on first failure = -x [OK]

Hint: Use -x to stop pytest after first failure [OK]

Common Mistakes:

• Using long option names that don't exist
• Confusing -f with -x
• Assuming --failfast=false stops tests

3. Given the command pytest -k 'test_add or test_subtract' -v, what will pytest do?

medium

A. Run all tests except those named test_add or test_subtract

B. Run only tests named test_add or test_subtract with verbose output

C. Run tests with names containing 'test' only

D. Run tests and stop after first failure

Solution

1. Step 1: Understand the -k option

   The -k option filters tests by substring expressions. Here, it selects tests named test_add or test_subtract.

2. Step 2: Understand the -v option

   The -v option runs tests in verbose mode, showing detailed test names and results.

3. Final Answer:

   Run only tests named test_add or test_subtract with verbose output -> Option B

4. Quick Check:

   -k filters tests, -v shows details [OK]

Hint: -k filters tests by name, -v shows details [OK]

Common Mistakes:

• Thinking -k excludes tests instead of including
• Ignoring the verbose flag
• Assuming it stops on failure

4. You run pytest --maxfail=2 -v but pytest does not stop after 2 failures. What is the likely cause?

medium

A. You need to add -x to stop after max failures

B. The option --maxfail requires a space before the number

C. The --maxfail option is deprecated and ignored

D. You must use --maxfail=2 without -v

Solution

1. Step 1: Understand --maxfail behavior

   The --maxfail option sets how many failures to allow before stopping, but it requires -x to actually stop the test run.

2. Step 2: Check command usage

   Without -x, pytest will report max failures but continue running all tests.

3. Final Answer:

   You need to add -x to stop after max failures -> Option A

4. Quick Check:

   --maxfail needs -x to stop tests [OK]

Hint: Add -x with --maxfail to stop tests early [OK]

Common Mistakes:

• Thinking --maxfail alone stops tests
• Adding space between --maxfail and number
• Assuming -v disables maxfail

5. You want to run pytest only on tests marked as slow and generate a JUnit XML report saved as results.xml. Which command is correct?

hard

A. pytest --only-mark slow --junitxml results.xml

B. pytest --mark slow --xml=results.xml

C. pytest -m slow --junitxml=results.xml

D. pytest -m slow -x --junit=results.xml

Solution

1. Step 1: Use -m to select tests by marker

   The -m option filters tests by marker name, so -m slow runs only tests marked as slow.

2. Step 2: Use --junitxml=filename to generate report

   The --junitxml=results.xml option saves test results in JUnit XML format to the specified file.

3. Step 3: Verify other options are invalid

   Options like --mark, --xml, and --only-mark are not valid pytest flags.

4. Final Answer:

   pytest -m slow --junitxml=results.xml -> Option C

5. Quick Check:

   -m for marker, --junitxml for report [OK]

Hint: Use -m for markers and --junitxml for reports [OK]

Common Mistakes:

• Using incorrect flags like --mark or --xml
• Forgetting = between --junitxml and filename
• Adding -x which stops tests unnecessarily