Test modules in PyTest - Deep Dive

Overview - Test modules

What is it?

Test modules are files that contain test functions or classes to check if your code works correctly. In pytest, these modules are Python files named with a test_ prefix or _test suffix. They organize tests into groups, making it easier to run and maintain them. Each test module can have multiple test cases that pytest discovers and runs automatically.

Why it matters

Without test modules, tests would be scattered and hard to find or run. This would make it difficult to verify that your code works as expected, leading to more bugs and slower development. Test modules help keep tests organized, so you can quickly check your code after changes and catch errors early. This saves time and improves software quality.

Where it fits

Before learning test modules, you should understand basic Python programming and how to write simple test functions. After mastering test modules, you can learn about test fixtures, parameterized tests, and test suites to build more powerful and reusable tests.

Mental Model

Core Idea

A test module is like a folder that neatly holds related tests so you can easily find, run, and manage them together.

Think of it like...

Imagine a test module as a recipe book where each recipe is a test. The book groups similar recipes so you can quickly find and try them all to see if they taste right.

┌─────────────────────────┐
│       Test Module       │
│  ┌───────────────┐      │
│  │ test_func_1() │      │
│  ├───────────────┤      │
│  │ test_func_2() │      │
│  ├───────────────┤      │
│  │ TestClass     │      │
│  │  ├ test_method│      │
│  │  └────────────┘      │
└─────────────────────────┘

Build-Up - 7 Steps

1

Foundation

What is a test module in pytest

Concept: Introduce the basic idea of a test module as a Python file containing tests.

A test module is a Python file named starting with test_ or ending with _test.py. It contains test functions or classes that pytest will find and run. For example, a file named test_math.py can have functions like test_addition() to check if addition works.

Result

Pytest recognizes the file as a test module and runs all test functions inside it.

Understanding that test modules are just Python files with special names helps you organize tests naturally using familiar file structures.

2

Foundation

Naming conventions for test modules

Concept: Explain the naming rules pytest uses to find test modules automatically.

Pytest looks for files matching test_*.py or *_test.py patterns. If your test file doesn't follow this, pytest will ignore it. For example, test_utils.py or utils_test.py are valid test modules, but utils.py is not.

Result

Only files with correct names are collected and run by pytest.

Knowing naming conventions prevents confusion when tests don't run and helps pytest find your tests without extra configuration.

3

Intermediate

Organizing multiple test modules

🤔Before reading on: Do you think putting all tests in one file or splitting into many files is better? Commit to your answer.

Concept: Show how to split tests into multiple modules for clarity and maintenance.

As your project grows, you create multiple test modules, each testing a part of your code. For example, test_math.py for math functions and test_strings.py for string functions. You can place these files in a tests/ folder to keep them separate from your main code.

Result

Tests are easier to find and maintain, and pytest runs all modules under the tests/ folder automatically.

Organizing tests into modules mirrors how your code is structured, making it easier to locate and fix problems.

4

Intermediate

Using test classes inside modules

🤔Before reading on: Do you think test classes can share setup code for multiple tests? Commit to your answer.

Concept: Introduce test classes as a way to group related tests and share setup code.

Inside a test module, you can define classes starting with Test, like class TestCalculator:. Inside, methods starting with test_ are individual tests. This groups tests logically and allows shared setup using special methods like setup_method().

Result

Tests inside the class run together, and setup code runs before each test method.

Using test classes helps avoid repeating code and keeps related tests bundled, improving readability and efficiency.

5

Intermediate

Running specific test modules

🤔Before reading on: Can you run only one test module without running all tests? Commit to your answer.

Concept: Teach how to run tests from a single module using pytest command options.

You can run tests from one module by specifying its file name: pytest tests/test_math.py. This runs only tests in that file, saving time when you want to focus on a specific area.

Result

Only tests in the chosen module execute, and pytest shows their results.

Knowing how to run specific modules speeds up debugging and testing during development.

6

Advanced

Test discovery and module import mechanics

🤔Before reading on: Do you think pytest imports test modules like normal Python modules or differently? Commit to your answer.

Concept: Explain how pytest finds and imports test modules to run tests.

Pytest searches folders recursively for files matching test patterns. It imports each test module as a normal Python module but with extra hooks to collect test functions and classes. If import fails due to errors, pytest reports it and skips that module.

Result

Pytest builds a list of test items from all discovered modules to run them in order.

Understanding import mechanics helps diagnose issues like import errors or missing tests during discovery.

7

Expert

Customizing test module collection

🤔Before reading on: Can you change which files pytest treats as test modules without renaming them? Commit to your answer.

Concept: Show how to customize pytest's test module discovery using configuration files.

You can configure pytest to recognize different file patterns by adding a pytest.ini file with the line: python_files = check_*.py custom_test.py. This lets you include or exclude files as test modules without renaming them.

Result

Pytest collects tests from files matching your custom patterns, giving flexibility in organizing tests.

Knowing how to customize test discovery allows integration with legacy code or special project structures.

Under the Hood

Pytest uses a test discovery process that scans directories for files matching naming patterns. It imports these files as Python modules, then inspects their contents to find test functions and classes by name patterns. Each test function or method is wrapped as a test item. Pytest then runs these items, capturing results and reporting them. Import errors or syntax errors in test modules cause pytest to report failures before running tests.

Why designed this way?

Pytest was designed to be simple and intuitive, using Python's own module system and naming conventions to find tests automatically. This avoids complex configuration and lets tests live alongside code naturally. The design balances ease of use with flexibility, allowing customization when needed. Alternatives like manual test registration were rejected to reduce boilerplate and errors.

┌───────────────┐
│  File System  │
│ (tests/ folder)│
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Test Discovery│
│ (find files)  │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Module Import │
│ (load files)  │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Test Collection│
│ (find tests)  │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Test Execution│
│ (run tests)   │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Do you think pytest runs any Python file with test functions, regardless of its name? Commit to yes or no.

Common Belief: Pytest runs all Python files that have functions starting with test_, no matter the file name.

Reality: Pytest only runs test functions in files named with test_*.py or *_test.py by default. Files with other names are ignored even if they have test functions.

Why it matters: If you name test files incorrectly, pytest will skip them silently, causing tests not to run and bugs to go unnoticed.

Quick: Do you think test classes must inherit from unittest.TestCase in pytest? Commit to yes or no.

Common Belief: Test classes in pytest must inherit from unittest.TestCase to work properly.

Reality: Pytest does not require test classes to inherit from any base class. It detects classes by name and runs their test methods automatically.

Why it matters: Assuming inheritance is required can lead to unnecessary complexity and confusion when writing pytest tests.

Quick: Do you think pytest runs tests in test modules in the order they appear in the file? Commit to yes or no.

Common Belief: Pytest runs tests in the order they are written in the test module.

Reality: Pytest collects tests in a sorted order by name, not by their order in the file. This means test execution order may differ from source order.

Why it matters: Relying on test order can cause flaky tests and hidden dependencies between tests, leading to unreliable test suites.

Quick: Do you think you can run tests from any file by just calling pytest on it, even if it doesn't follow naming conventions? Commit to yes or no.

Common Belief: You can run tests from any Python file by specifying it to pytest, regardless of its name.

Reality: Pytest ignores files that don't match test file patterns, even if you specify them. You must rename or configure pytest to include them.

Why it matters: Trying to run tests from wrongly named files wastes time and causes confusion about why tests don't run.

Expert Zone

1

Pytest's test module discovery respects __init__.py files, affecting whether directories are treated as packages, which can influence import paths and test discovery.

2

Test modules can be dynamically generated or modified at runtime using hooks, allowing advanced customization of test collection and execution.

3

Pytest caches test discovery results to speed up repeated runs, but stale caches can cause confusing test runs if files are renamed or moved.

When NOT to use

Test modules are not suitable when you need to test code in non-Python environments or when tests require complex setup better handled by integration or system testing frameworks. In such cases, use specialized tools like Selenium for UI tests or Postman for API tests.

Production Patterns

In real projects, test modules are organized mirroring the source code structure, often under a tests/ directory. Teams use pytest markers to group tests across modules, and continuous integration systems run test modules in parallel to speed up feedback. Test modules are also used with coverage tools to measure how much code is tested.

Connections

Modular programming

Test modules build on the idea of modular programming by organizing tests into separate files.

Understanding modular programming helps grasp why grouping tests into modules improves maintainability and clarity.

File system organization

Test modules rely on file system naming and structure to enable automatic discovery.

Knowing how file systems and naming conventions work helps understand how pytest finds and runs tests.

Scientific experiment design

Test modules are like experiment batches where each module tests a hypothesis about code behavior.

Seeing tests as experiments clarifies why grouping related tests helps isolate and understand failures.

Common Pitfalls

#1 Tests not running because test module is misnamed

Wrong approach:

File named utils.py containing test functions like test_addition()

Correct approach:

Rename file to test_utils.py so pytest recognizes it as a test module

Root cause:

Misunderstanding pytest's naming conventions for test discovery

#2 Assuming test classes must inherit from unittest.TestCase

Wrong approach:

class TestMath(unittest.TestCase):
def test_add(self):
assert 1 + 1 == 2

Correct approach:

class TestMath:
def test_add(self):
assert 1 + 1 == 2

Root cause:

Confusing pytest with unittest framework requirements

#3 Relying on test execution order inside a module

Wrong approach:

Writing tests that depend on previous tests running first, expecting order to be source order

Correct approach:

Design tests to be independent and not rely on execution order, since pytest runs tests sorted by name

Root cause:

Not knowing pytest's test collection order and test independence best practices

Key Takeaways

Test modules are Python files named with test_ or _test.py that group related tests for easy discovery and execution by pytest.

Following pytest's naming conventions is essential to ensure your tests are found and run automatically.

Organizing tests into multiple modules and using test classes improves clarity, maintenance, and code reuse.

Pytest imports test modules as normal Python modules but adds hooks to collect and run tests efficiently.

Customizing test module discovery and understanding pytest's internals empowers you to handle complex project structures and debugging.

Practice

1. What is the correct way to name a Python file so that pytest recognizes it as a test module?

easy

A. The file name can be anything as long as it contains tests.

B. The file name should start with check_.

C. The file name should be pytest_module.py.

D. The file name should start with test_ or end with _test.py.

Solution

1. Step 1: Understand pytest file naming conventions

   pytest automatically discovers test files if their names start with test_ or end with _test.py.

2. Step 2: Evaluate each option

   Only The file name should start with test_ or end with _test.py. matches the correct naming pattern for pytest to find test modules automatically.

3. Final Answer:

   The file name should start with test_ or end with _test.py. -> Option D

4. Quick Check:

   pytest test file naming = start with test_ or end with _test.py [OK]

Hint: Remember test files start with test_ or end with _test.py [OK]

Common Mistakes:

• Assuming any file with tests is recognized
• Using incorrect prefixes like check_
• Naming files without .py extension

2. Which of the following is the correct syntax for a test function inside a pytest test module?

easy

A. def test_function():

B. def check_function():

C. def function_test():

D. def testing_function():

Solution

1. Step 1: Recall pytest test function naming rules

   pytest runs functions that start with test_ automatically as tests.

2. Step 2: Match options with naming rule

   Only def test_function(): defines a function starting with test_, so pytest will recognize it as a test.

3. Final Answer:

   def test_function(): -> Option A

4. Quick Check:

   Test functions start with test_ [OK]

Hint: Test functions must start with test_ to run automatically [OK]

Common Mistakes:

• Naming test functions without test_ prefix
• Using other prefixes like check_ or testing_
• Forgetting to define functions with def

3. Given the following test module test_sample.py with these functions:

def test_add():
    assert 1 + 1 == 2

def add():
    return 1 + 1

def test_subtract():
    assert 2 - 1 == 1

Which functions will pytest run as tests?

medium

A. All three functions

B. Only add

C. test_add and test_subtract

D. Only test_add

Solution

1. Step 1: Identify pytest test function naming

   pytest runs functions starting with test_ automatically as tests.

2. Step 2: Check each function name

   test_add and test_subtract start with test_, so pytest runs them. add does not start with test_, so it is ignored.

3. Final Answer:

   test_add and test_subtract -> Option C

4. Quick Check:

   Functions starting with test_ run as tests [OK]

Hint: Only functions starting with test_ run as tests [OK]

Common Mistakes:

• Thinking all functions in test file run as tests
• Confusing helper functions with test functions
• Ignoring function name prefixes

4. You have a test module named test_math.py with this code:

def test_multiply():
    assert 2 * 3 == 5

When you run pytest, what will happen?

medium

A. The test will fail because the assertion is incorrect.

B. pytest will not find any tests to run.

C. pytest will skip the test because of a syntax error.

D. The test will pass because 2 * 3 equals 6.

Solution

1. Step 1: Analyze the assertion in test_multiply

   The assertion checks if 2 * 3 == 5, which is false because 2 * 3 equals 6.

2. Step 2: Understand pytest test result

   pytest runs the test and finds the assertion false, so the test fails.

3. Final Answer:

   The test will fail because the assertion is incorrect. -> Option A

4. Quick Check:

   False assertion causes test failure [OK]

Hint: Check if assertion condition is true or false [OK]

Common Mistakes:

• Assuming test passes despite wrong assertion
• Confusing syntax errors with assertion failures
• Thinking pytest skips failing tests

5. You want to organize tests for a calculator app. You create two files: test_add.py and test_subtract.py. Each file has multiple test functions starting with test_. How does pytest handle these test modules when you run pytest in the folder?

hard

A. pytest requires you to specify each test module to run tests.

B. pytest runs all test functions from both modules automatically.

C. pytest runs only tests from the first module alphabetically.

D. pytest runs tests only if you import modules manually.

Solution

1. Step 1: Understand pytest test discovery

   pytest automatically finds all test modules named with test_ prefix and runs all test functions inside them.

2. Step 2: Apply to multiple test modules

   When running pytest in the folder, it collects tests from both test_add.py and test_subtract.py and runs all test functions starting with test_.

3. Final Answer:

   pytest runs all test functions from both modules automatically. -> Option B

4. Quick Check:

   pytest auto-discovers all test modules and functions [OK]

Hint: pytest runs all test_ modules and functions in folder automatically [OK]

Common Mistakes:

• Thinking pytest runs only one module at a time
• Believing manual imports are needed for tests
• Assuming tests must be specified explicitly