Running tests (pytest command) - Framework Patterns

Framework Mode - Running tests (pytest command)

Folder Structure

tests/
├── test_example.py
├── test_login.py
├── test_checkout.py
conftest.py
pytest.ini
requirements.txt

This is a simple pytest project structure. All test files are inside the tests/ folder. The conftest.py file holds shared fixtures and hooks. The pytest.ini file contains pytest configuration.

Test Framework Layers

• Test files: Contain test functions or classes starting with test_. These are the actual tests pytest runs.
• Fixtures (conftest.py): Setup and teardown code shared across tests, like preparing test data or opening a browser.
• Helpers/Utilities: Optional Python modules with reusable functions to keep tests clean.
• Configuration: Files like pytest.ini to customize pytest behavior.

Configuration Patterns

• pytest.ini file to set default options, e.g., test paths, markers, or verbosity.
• Use environment variables or pytest command-line options to select environments or browsers.
• conftest.py fixtures can read config values and provide them to tests.
• Example pytest.ini content:

  [pytest]
  addopts = -v --maxfail=2
  markers =
      smoke: quick smoke tests
      regression: full regression suite
  

Test Reporting and CI/CD Integration

• Pytest outputs test results in the console by default (pass/fail with details).
• Use plugins like pytest-html to generate HTML reports for easy reading.
• Integrate pytest commands in CI/CD pipelines (GitHub Actions, Jenkins) to run tests automatically on code changes.
• Example command to run tests and generate HTML report:

  pytest tests/ --html=report.html --self-contained-html

Best Practices

• Keep test names descriptive and start with test_ so pytest can find them easily.
• Use pytest.ini to avoid repeating command-line options every time.
• Run tests often during development to catch issues early.
• Use markers to group tests and run subsets (e.g., smoke tests) quickly.
• Keep tests independent so they can run in any order without failing.

Self Check

Where in this framework structure would you add a new fixture to open a browser before tests?

Key Result

Use pytest command to run tests organized in the tests/ folder with configuration in pytest.ini and shared fixtures in conftest.py.

Practice

1. What does the basic pytest command do when run in a project folder?

easy

A. It only checks the syntax of test files without running them.

B. It automatically finds and runs all test files and test functions.

C. It deletes all test files in the folder.

D. It installs pytest in the project folder.

Solution

1. Step 1: Understand the default behavior of pytest command

   Running pytest without arguments searches for test files and runs all tests found.

2. Step 2: Eliminate incorrect options

   Options A, C, and D describe actions pytest does not perform by default.

3. Final Answer:

   It automatically finds and runs all test files and test functions. -> Option B

4. Quick Check:

   pytest runs tests automatically = B [OK]

Hint: pytest runs all tests by default when no file is specified [OK]

Common Mistakes:

• Thinking pytest only checks syntax
• Assuming pytest deletes files
• Confusing pytest with package installer

2. Which of the following is the correct command to run pytest with detailed output?

easy

A. pytest -v

B. pytest --detail

C. pytest -d

D. pytest --verbose-output

Solution

1. Step 1: Recall pytest option for verbose output

   The -v option stands for verbose and shows detailed test results.

2. Step 2: Check other options for correctness

   Options -d, --detail, and --verbose-output are not valid pytest flags.

3. Final Answer:

   pytest -v -> Option A

4. Quick Check:

   Verbose flag is -v = C [OK]

Hint: Use -v for verbose pytest output [OK]

Common Mistakes:

• Using -d instead of -v
• Typing long incorrect flags
• Confusing verbose with debug flags

3. Given the command pytest test_math.py, what will happen?

medium

A. pytest runs all tests in all files in the folder.

B. pytest runs tests but ignores test_math.py.

C. pytest runs only the tests inside the file named test_math.py.

D. pytest runs tests only if they are named test_math.py inside any folder.

Solution

1. Step 1: Understand pytest file argument behavior

   When a filename is given, pytest runs tests only from that file.

2. Step 2: Eliminate incorrect options

   A: Runs all tests in all files (wrong when file specified). B: Ignores test_math.py (wrong). C: Misunderstands filename as test name search (wrong).

3. Final Answer:

   pytest runs only the tests inside the file named test_math.py. -> Option C

4. Quick Check:

   pytest filename runs tests in that file only = D [OK]

Hint: Specify filename to run tests only from that file [OK]

Common Mistakes:

• Assuming pytest runs all tests ignoring filename
• Confusing filename with test function name
• Thinking pytest ignores specified files

4. You run pytest -v test_example.py but get an error: pytest: error: unrecognized arguments: -v. What is the likely cause?

medium

A. pytest does not support the -v option.

B. You typed the option incorrectly; it should be --verbose.

C. The test file test_example.py does not exist.

D. You are running pytest with a wrong command or from a wrong environment.

Solution

1. Step 1: Understand the error message

   The error says -v is unrecognized, which is unusual because -v is a valid pytest option.

2. Step 2: Identify environment or command issues

   This usually means pytest is not properly installed or a different command is running (like python or another tool).

3. Final Answer:

   You are running pytest with a wrong command or from a wrong environment. -> Option D

4. Quick Check:

   Unrecognized option means wrong environment = A [OK]

Hint: Check pytest installation and environment if options fail [OK]

Common Mistakes:

• Assuming -v is invalid option
• Ignoring environment issues
• Blaming test file existence for option errors

5. You want to run only tests marked as slow in your project using pytest. Which command correctly runs only those tests?

hard

A. pytest -m slow

B. pytest --only slow

C. pytest --mark slow

D. pytest -s slow

Solution

1. Step 1: Recall pytest marker option

   The -m option runs tests matching a marker expression, such as slow.

2. Step 2: Verify other options

   Options --only, --mark, and -s do not filter tests by marker.

3. Final Answer:

   pytest -m slow -> Option A

4. Quick Check:

   Use -m to run tests by marker = A [OK]

Hint: Use -m marker to run tests with that marker [OK]

Common Mistakes:

• Using incorrect flags like --only or --mark
• Confusing -s (capture output) with marker filtering
• Not marking tests before filtering