Matching exception messages in PyTest - Framework Patterns

Framework Mode - Matching exception messages

Folder Structure

tests/
├── test_example.py
├── test_utils.py
├── conftest.py
utilities/
├── helpers.py
configs/
├── config.yaml
reports/
├── latest_report.html

Test Framework Layers

• Tests Layer: Contains test files like test_example.py where tests assert exception messages using pytest.raises().
• Utilities Layer: Helper functions or custom assertion utilities to check exception messages precisely.
• Configuration Layer: Holds environment and test settings in configs/config.yaml or conftest.py fixtures.
• Reporting Layer: Generates test reports after execution, showing pass/fail and exception details.

Configuration Patterns

• Use conftest.py to define fixtures for test setup and teardown.
• Store environment-specific data (like error message templates) in configs/config.yaml.
• Use pytest command line options or environment variables to select test environments or verbosity.
• Configure pytest.ini or pyproject.toml for pytest options like markers and logging.

Test Reporting and CI/CD Integration

• Use pytest built-in reporting with --tb=short for concise tracebacks.
• Integrate with plugins like pytest-html to generate readable HTML reports showing exception messages.
• Configure CI pipelines (GitHub Actions, Jenkins) to run tests and publish reports automatically.
• Fail tests clearly when exception messages do not match expected patterns.

Best Practices

1. Use pytest.raises() context manager to catch exceptions and assert their messages.
2. Match exception messages using substring checks or regular expressions for flexibility.
3. Keep error message expectations in test code or external config for maintainability.
4. Write clear, descriptive assertion messages to help debugging when tests fail.
5. Use fixtures to share common setup for tests that check exceptions.

Self Check

Where in this folder structure would you add a helper function to validate exception messages with regex?

Key Result

Use pytest.raises() with message matching and clear folder layers for maintainable exception message testing.

Practice

1. What is the main purpose of using pytest.raises() in a test?

easy

A. To check if a specific exception is raised during code execution

B. To skip a test when an exception occurs

C. To print the exception message to the console

D. To ignore exceptions and continue the test

Solution

1. Step 1: Understand pytest.raises() usage

   The pytest.raises() function is used to verify that a block of code raises a specific exception.

2. Step 2: Purpose of pytest.raises()

   It helps confirm that the code fails as expected by raising the correct error, which is important for testing error handling.

3. Final Answer:

   To check if a specific exception is raised during code execution -> Option A

4. Quick Check:

   pytest.raises() checks exceptions = D [OK]

Hint: pytest.raises() confirms expected errors occur [OK]

Common Mistakes:

• Thinking pytest.raises() skips tests
• Assuming it prints exceptions automatically
• Believing it ignores exceptions

2. Which of the following is the correct syntax to check an exception message contains the word 'error' using pytest?

easy

A. with pytest.raises(ValueError) as exc_info:
assert 'error' in exc_info.value

B. with pytest.raises(ValueError) as exc_info:
assert exc_info.message == 'error'

C. with pytest.raises(ValueError) as exc_info:
assert exc_info.error == 'error'

D. with pytest.raises(ValueError) as exc_info:
assert 'error' in str(exc_info.value)

Solution

1. Step 1: Understand exc_info object

   The exc_info object stores the caught exception. To get its message as a string, use str(exc_info.value).

2. Step 2: Correct assertion syntax

   Checking if 'error' is in the exception message requires assert 'error' in str(exc_info.value). Directly accessing exc_info.value without converting to string won't work.

3. Final Answer:

   with pytest.raises(ValueError) as exc_info: assert 'error' in str(exc_info.value) -> Option D

4. Quick Check:

   Exception message check uses str(exc_info.value) = C [OK]

Hint: Convert exc_info.value to string before checking message [OK]

Common Mistakes:

• Forgetting to convert exc_info.value to string
• Using exc_info.message which does not exist
• Comparing exc_info.error instead of message

3. What will be the result of this pytest test code?

def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError) as exc_info:
        x = 1 / 0
    assert 'division by zero' in str(exc_info.value)

medium

A. Test fails because exception message does not contain 'division by zero'

B. Test fails due to syntax error in the test code

C. Test passes because ZeroDivisionError is raised with correct message

D. Test passes but assertion is ignored

Solution

1. Step 1: Check exception raised

   The code divides 1 by 0, which raises a ZeroDivisionError as expected.

2. Step 2: Verify exception message content

   The message for ZeroDivisionError in Python includes 'division by zero', so the assertion passes.

3. Final Answer:

   Test passes because ZeroDivisionError is raised with correct message -> Option C

4. Quick Check:

   ZeroDivisionError message contains 'division by zero' = B [OK]

Hint: Check actual exception message text matches assertion [OK]

Common Mistakes:

• Assuming exception message is different
• Thinking assertion is ignored
• Confusing syntax error with runtime error

4. Identify the error in this pytest test code snippet:

def test_key_error():
    with pytest.raises(KeyError) as exc_info:
        d = {'a': 1}
        value = d['b']
    assert 'b' in exc_info.value

medium

A. The assertion should convert exc_info.value to string before checking

B. The exception type should be ValueError, not KeyError

C. The dictionary key 'b' exists, so no exception is raised

D. pytest.raises() cannot catch KeyError exceptions

Solution

1. Step 1: Check exception type and code

   Accessing d['b'] raises a KeyError because 'b' is not in the dictionary.

2. Step 2: Analyze assertion correctness

   The assertion checks if 'b' is in exc_info.value directly, but exc_info.value is an exception object, not a string. It should be converted to string first.

3. Final Answer:

   The assertion should convert exc_info.value to string before checking -> Option A

4. Quick Check:

   Always use str(exc_info.value) for message checks = A [OK]

Hint: Always convert exc_info.value to string before assertions [OK]

Common Mistakes:

• Checking exception object directly without str()
• Using wrong exception type
• Assuming key exists in dictionary

5. You want to test a function that raises a ValueError with message "Invalid input: 0" when input is zero. Which pytest test code correctly verifies both exception type and exact message?

hard

A. with pytest.raises(ValueError):
func(0)
assert 'Invalid input' in exc_info.value

B. with pytest.raises(ValueError) as exc_info:
func(0)
assert exc_info.value.args[0] == 'Invalid input: 0'

C. with pytest.raises(ValueError) as exc_info:
func(0)
assert exc_info.value.message == 'Invalid input: 0'

D. with pytest.raises(ValueError) as exc_info:
func(0)
assert str(exc_info.value) == 'ValueError: Invalid input: 0'

Solution

1. Step 1: Understand exception message access

   Exception messages are stored in args tuple; the first element args[0] holds the message string.

2. Step 2: Verify correct assertion for exact message

   Using exc_info.value.args[0] allows exact message comparison. Using str(exc_info.value) works but may include extra formatting. exc_info.value.message does not exist.

3. Final Answer:

   with pytest.raises(ValueError) as exc_info: func(0) assert exc_info.value.args[0] == 'Invalid input: 0' -> Option B

4. Quick Check:

   Use exc_info.value.args[0] for exact exception message = A [OK]

Hint: Use exc_info.value.args[0] for exact message match [OK]

Common Mistakes:

• Using exc_info.value.message which is invalid
• Checking message without exc_info context
• Assuming str() always matches exactly