setup.cfg configuration in PyTest - Build an Automation Script

Configure pytest options using setup.cfg and verify test discovery

Preconditions (2)

Step 1: Create a setup.cfg file in the project root

Step 2: Add pytest configuration to setup.cfg to specify test paths and markers

Step 3: Run pytest from the command line without additional arguments

Step 4: Observe that pytest discovers and runs tests according to setup.cfg settings

✅ Expected Result: Pytest runs tests found in the configured test paths and applies any specified markers or options from setup.cfg

Automation Requirements - pytest

Assertions Needed:

Verify pytest discovers the test_sample.py test file

Verify the test function inside test_sample.py runs and passes

Verify pytest output includes configured markers or options

Best Practices:

Use setup.cfg for pytest configuration instead of command line flags

Keep test files and functions named according to pytest conventions

Use pytest's capsys or caplog fixtures to capture output for assertions

Automated Solution

PyTest

[pytest]
testpaths = tests
markers =
    smoke: mark a test as smoke test

# test_sample.py file content

def test_example():
    assert 1 + 1 == 2

# Command to run in terminal:
# pytest

# Expected output includes:
# - test discovery in 'tests' folder
# - test_example passes

The setup.cfg file is configured with a [pytest] section. It sets testpaths = tests so pytest looks for tests inside the tests folder. It also defines a custom marker smoke for demonstration.

The test_sample.py file contains a simple test function test_example that asserts 1 + 1 equals 2.

Running pytest in the terminal will use the setup.cfg configuration automatically. Pytest will discover tests in the tests directory and run test_example. The test should pass, confirming the configuration works.

This approach avoids passing command line options every time and centralizes test configuration.

Common Mistakes - 3 Pitfalls

{'mistake': 'Placing pytest configuration in the wrong section or file', 'why_bad': "Pytest will ignore configuration if it's not under the correct [pytest] section in setup.cfg", 'correct_approach': 'Always put pytest options under the [pytest] section in setup.cfg'}

{'mistake': 'Naming test files or functions incorrectly', 'why_bad': "Pytest won't discover tests if files/functions don't follow naming conventions (e.g., test_*.py, test_*)", 'correct_approach': "Name test files starting with 'test_' and test functions starting with 'test_'"}

Running pytest with conflicting command line options overriding setup.cfg

Bonus Challenge

Now add a custom marker 'smoke' in setup.cfg and write a test that uses this marker. Run pytest to verify the marker is recognized.

Show Hint

Practice

1. What is the main purpose of the setup.cfg file in pytest?

easy

A. To store pytest configuration settings in one place

B. To write test functions

C. To install pytest package

D. To run tests manually from the command line

Solution

1. Step 1: Understand pytest configuration

   The setup.cfg file is used to keep pytest settings centralized.

2. Step 2: Differentiate from other tasks

   Writing tests or installing packages are not roles of setup.cfg.

3. Final Answer:

   To store pytest configuration settings in one place -> Option A

4. Quick Check:

   setup.cfg centralizes pytest settings [OK]

Hint: Remember setup.cfg holds config, not test code [OK]

Common Mistakes:

• Confusing setup.cfg with test files
• Thinking setup.cfg runs tests
• Assuming setup.cfg installs packages

2. Which of the following is the correct way to specify the test file pattern in setup.cfg?

easy

A. [pytest]\ntestpaths = tests

B. [pytest]\npython_files = test_*.py

C. [tool:pytest]\npython_files = *.txt

D. [pytest]\nmarkers = test_*.py

Solution

1. Step 1: Identify correct section and key

   pytest settings go under [pytest] and python_files sets test file patterns.

2. Step 2: Check pattern correctness

   python_files = test_*.py matches typical test files.

3. Final Answer:

   [pytest]\npython_files = test_*.py -> Option B

4. Quick Check:

   python_files sets test file pattern [OK]

Hint: Use python_files to set test file patterns [OK]

Common Mistakes:

• Using wrong section header like [tool:pytest]
• Confusing markers with file patterns
• Setting testpaths instead of python_files for file pattern

3. Given this setup.cfg snippet:

[pytest]
markers = slow: marks tests as slow
python_files = test_*.py

What happens when you run pytest -m slow?

medium

A. Tests in files not matching test_*.py run

B. All tests run ignoring markers

C. Tests with names containing 'slow' run

D. Only tests marked with @pytest.mark.slow run

Solution

1. Step 1: Understand marker definition

   The marker slow is defined to label slow tests.

2. Step 2: Effect of -m slow option

   Running pytest -m slow runs only tests marked with @pytest.mark.slow.

3. Final Answer:

   Only tests marked with @pytest.mark.slow run -> Option D

4. Quick Check:

   -m runs tests with matching marker [OK]

Hint: -m runs tests with matching marker only [OK]

Common Mistakes:

• Thinking -m filters by test name
• Assuming all tests run ignoring markers
• Confusing file pattern with marker filtering

4. You added this to setup.cfg:

[pytest]
markers = slow: marks slow tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

But pytest gives a warning about unknown markers. What is the likely cause?

medium

A. Markers need to be registered with full description and no typos

B. Missing a blank line after [pytest] section

C. Markers must be listed under [tool:pytest] section

D. python_files setting conflicts with markers

Solution

1. Step 1: Check marker registration format

   Markers must be registered exactly with correct spelling and description.

2. Step 2: Understand warning cause

   Warnings usually occur if marker name is misspelled or description is missing.

3. Final Answer:

   Markers need to be registered with full description and no typos -> Option A

4. Quick Check:

   Correct marker registration avoids warnings [OK]

Hint: Check marker spelling and description carefully [OK]

Common Mistakes:

• Placing markers under wrong section
• Forgetting marker description after colon
• Assuming python_files affects markers

5. You want pytest to automatically ignore tests in the legacy_tests folder without typing command line options. How do you configure setup.cfg to do this?

hard

A. [pytest]
markers = ignore: skip legacy tests

B. [pytest]
python_files = test_*.py
ignore = legacy_tests

C. [pytest]
addopts = --ignore=legacy_tests

D. [pytest]
filterwarnings = ignore::legacy_tests

Solution

1. Step 1: Identify option to ignore folders

   The --ignore=folder option tells pytest to skip that folder.

2. Step 2: Set addopts in setup.cfg

   Adding addopts = --ignore=legacy_tests runs pytest with this option automatically.

3. Final Answer:

   [pytest]\naddopts = --ignore=legacy_tests -> Option C

4. Quick Check:

   addopts sets default command line options [OK]

Hint: Use addopts to add default command line options [OK]

Common Mistakes:

• Using ignore key instead of addopts
• Confusing markers with ignore settings
• Using filterwarnings incorrectly for ignoring tests