Test packages in PyTest - Build an Automation Script

Verify pytest discovers and runs tests in a test package

Preconditions (4)

Step 1: Open a terminal or command prompt

Step 2: Navigate to the project root directory (the parent of 'tests' folder)

Step 3: Run the command 'pytest tests'

Step 4: Observe the test discovery and execution output

✅ Expected Result: pytest discovers all test functions in both 'test_math.py' and 'test_string.py' inside the 'tests' package and runs them, showing a summary with all tests passed

Automation Requirements - pytest

Assertions Needed:

Verify that pytest discovers all test functions in the 'tests' package

Verify that all tests run and pass successfully

Best Practices:

Use a test package (folder with __init__.py) to organize tests

Name test files and functions with 'test_' prefix for pytest discovery

Use pytest's built-in assertion rewriting for clear test assertions

Keep tests independent and simple

Automated Solution

PyTest

import subprocess
import sys
import os

def test_pytest_discovers_and_runs_tests():
    # Run pytest on the 'tests' package
    result = subprocess.run([sys.executable, '-m', 'pytest', 'tests', '--tb=short', '--disable-warnings'], capture_output=True, text=True)
    output = result.stdout
    # Check that pytest ran tests from both test modules
    assert 'test_math.py' in output, "test_math.py tests not found in pytest output"
    assert 'test_string.py' in output, "test_string.py tests not found in pytest output"
    # Check that all tests passed
    assert 'failed' not in output.lower(), "Some tests failed"
    assert 'error' not in output.lower(), "Some tests had errors"
    assert 'passed' in output.lower(), "No tests passed"
    # Check that pytest exited with code 0
    assert result.returncode == 0, f"pytest exited with code {result.returncode}"

This test runs pytest as a subprocess on the 'tests' package folder.

It captures the output and checks that both test modules appear in the output, meaning pytest discovered them.

It asserts that no tests failed or errored, and that some tests passed.

Finally, it asserts that pytest exited with code 0, indicating success.

This approach tests the package discovery and execution end-to-end, simulating a real user running pytest.

Common Mistakes - 4 Pitfalls

{'mistake': "Not naming test files or functions with 'test_' prefix", 'why_bad': "pytest will not discover tests without the 'test_' prefix, so tests won't run", 'correct_approach': "Always name test files and functions starting with 'test_' to ensure pytest finds them"}

Running pytest without specifying the test package folder

Not checking pytest exit code in automation

Parsing pytest output with fragile string matching

Bonus Challenge

Now add data-driven testing with 3 different inputs in one of the test modules

Show Hint

Practice

1. What is the main purpose of a test package in pytest?

easy

A. To group related test files into a folder with an __init__.py file

B. To run tests only from a single test file

C. To create a virtual environment for tests

D. To install pytest dependencies automatically

Solution

1. Step 1: Understand test package structure

   A test package groups related test files inside a folder that contains an __init__.py file.

2. Step 2: Purpose of grouping tests

   This grouping helps organize tests and allows pytest to find and run them together.

3. Final Answer:

   To group related test files into a folder with an __init__.py file -> Option A

4. Quick Check:

   Test packages group tests = A [OK]

Hint: Test packages group tests in folders with __init__.py [OK]

Common Mistakes:

• Confusing test packages with virtual environments
• Thinking test packages run only one file
• Believing pytest installs dependencies automatically

2. Which of the following is the correct way to make a folder a test package in pytest?

easy

A. Add a file named test_package.py inside the folder

B. Create a file named setup.py inside the folder

C. Add a file named __init__.py inside the folder

D. Add a file named requirements.txt inside the folder

Solution

1. Step 1: Identify the marker for Python packages

   Python treats folders with an __init__.py file as packages, including test packages.

2. Step 2: Confirm pytest package recognition

   pytest uses this file to recognize the folder as a test package and find tests inside it.

3. Final Answer:

   Add a file named __init__.py inside the folder -> Option C

4. Quick Check:

   __init__.py marks test packages = B [OK]

Hint: Add __init__.py to folder to make it a test package [OK]

Common Mistakes:

• Adding test files instead of __init__.py to mark package
• Confusing setup.py with package marker
• Using requirements.txt for package identification

3. Given this folder structure:

tests/
  __init__.py
  test_math.py
  test_string.py

What will happen when you run pytest tests/?

medium

A. pytest will skip all tests because folder has __init__.py

B. pytest will run only test_math.py tests

C. pytest will raise an error because __init__.py is present

D. pytest will run all tests in both test_math.py and test_string.py

Solution

1. Step 1: Understand pytest test discovery in packages

   pytest finds and runs all test files inside a test package folder when you run pytest on that folder.

2. Step 2: Effect of __init__.py on test discovery

   The presence of __init__.py marks the folder as a package but does not prevent pytest from running tests inside.

3. Final Answer:

   pytest will run all tests in both test_math.py and test_string.py -> Option D

4. Quick Check:

   pytest runs all tests in package folder = D [OK]

Hint: pytest runs all tests in package folder with __init__.py [OK]

Common Mistakes:

• Thinking __init__.py blocks test discovery
• Assuming only one test file runs
• Expecting errors due to __init__.py presence

4. You have a folder tests/ with test files but forgot to add __init__.py. What issue might you face?

medium

A. pytest will not recognize the folder as a test package and may skip tests inside

B. pytest will raise a syntax error when running tests

C. pytest will run tests twice

D. pytest will automatically create __init__.py for you

Solution

1. Step 1: Role of __init__.py in test packages

   Without __init__.py, the folder is not a recognized Python package, so pytest may not treat it as a test package.

2. Step 2: Effect on test discovery

   pytest might skip tests inside such folders or not group them properly during test runs.

3. Final Answer:

   pytest will not recognize the folder as a test package and may skip tests inside -> Option A

4. Quick Check:

   Missing __init__.py can cause pytest to skip tests = C [OK]

Hint: Always add __init__.py to test folders to avoid skipped tests [OK]

Common Mistakes:

• Expecting syntax errors without __init__.py
• Thinking pytest auto-creates __init__.py
• Assuming tests run twice without __init__.py

5. You want to organize tests for a large project using pytest. You create nested folders with __init__.py files like this:

tests/
  __init__.py
  unit/
    __init__.py
    test_utils.py
  integration/
    __init__.py
    test_api.py

How can you run only the integration tests using pytest?

hard

A. Run pytest tests/ and pytest will run only integration tests automatically

B. Run pytest tests/integration/ to run only integration tests

C. Run pytest tests/unit/ to run integration tests

D. Remove __init__.py files to run only integration tests

Solution

1. Step 1: Understand folder-based test selection

   pytest runs tests in the folder path you specify. To run only integration tests, specify the integration folder.

2. Step 2: Confirm effect of nested __init__.py files

   Nested __init__.py files mark subfolders as packages but do not affect running tests by folder path.

3. Final Answer:

   Run pytest tests/integration/ to run only integration tests -> Option B

4. Quick Check:

   Run pytest on specific folder to run its tests = A [OK]

Hint: Run pytest on specific test folder to run only those tests [OK]

Common Mistakes:

• Expecting pytest to auto-filter integration tests without folder path
• Running wrong folder path for desired tests
• Removing __init__.py thinking it filters tests