Log level filtering in PyTest - Build an Automation Script

Verify log messages are filtered by log level

Preconditions (2)

Step 1: Configure the logger to capture logs at INFO level

Step 2: Call the function that logs messages at DEBUG, INFO, WARNING, and ERROR levels

Step 3: Capture the logs during the function execution

Step 4: Verify that DEBUG messages are not present in the captured logs

Step 5: Verify that INFO, WARNING, and ERROR messages are present in the captured logs

✅ Expected Result: Only INFO, WARNING, and ERROR log messages are captured; DEBUG messages are filtered out

Automation Requirements - pytest

Assertions Needed:

Assert DEBUG messages are not in captured logs

Assert INFO messages are in captured logs

Assert WARNING messages are in captured logs

Assert ERROR messages are in captured logs

Best Practices:

Use pytest's caplog fixture to capture logs

Set logger level explicitly before test

Use clear and specific assertions for log content

Avoid hardcoding log message strings; use variables or constants

Automated Solution

PyTest

import logging
import pytest

# Function that logs messages at different levels
def log_messages():
    logger = logging.getLogger('test_logger')
    logger.debug('Debug message')
    logger.info('Info message')
    logger.warning('Warning message')
    logger.error('Error message')


def test_log_level_filtering(caplog):
    logger = logging.getLogger('test_logger')
    logger.setLevel(logging.INFO)  # Set log level to INFO

    with caplog.at_level(logging.INFO, logger='test_logger'):
        log_messages()

    # Extract captured log messages
    messages = [record.getMessage() for record in caplog.records]

    # Assertions
    assert 'Debug message' not in messages, 'DEBUG messages should be filtered out'
    assert 'Info message' in messages, 'INFO message should be captured'
    assert 'Warning message' in messages, 'WARNING message should be captured'
    assert 'Error message' in messages, 'ERROR message should be captured'

The log_messages function logs messages at four levels.

In the test test_log_level_filtering, we set the logger level to INFO to filter out DEBUG messages.

We use pytest's caplog fixture to capture logs during the function call.

After running the function, we collect all log messages and assert that DEBUG messages are not present, while INFO, WARNING, and ERROR messages are present.

This approach ensures the log level filtering works as expected.

Common Mistakes - 4 Pitfalls

Not setting the logger level explicitly before the test

Using print statements instead of logging

{'mistake': 'Not using caplog fixture and trying to capture logs manually', 'why_bad': "Manual capturing is error-prone and more complex than using pytest's built-in caplog fixture.", 'correct_approach': 'Use caplog fixture to capture logs cleanly and reliably.'}

Hardcoding log message strings in assertions without matching exact messages

Bonus Challenge

Now add data-driven testing with 3 different logger levels: DEBUG, INFO, WARNING. Verify that only messages at or above the set level are captured.

Show Hint

Practice

1. What does caplog.at_level() do in pytest?

easy

A. It sets the minimum log level to capture during a test.

B. It clears all captured logs before the test runs.

C. It disables logging for the entire test session.

D. It changes the log message format.

Solution

1. Step 1: Understand caplog usage

   caplog is a pytest fixture used to capture log messages during tests.

2. Step 2: Purpose of at_level()

   The method at_level() sets the minimum log level to capture, filtering out lower-level messages.

3. Final Answer:

   It sets the minimum log level to capture during a test. -> Option A

4. Quick Check:

   caplog.at_level() sets log capture level [OK]

Hint: Remember: at_level sets minimum log level to capture [OK]

Common Mistakes:

• Thinking it clears logs instead of setting level
• Confusing it with disabling logging
• Assuming it changes log format

2. Which of the following is the correct syntax to capture only WARNING and above logs in pytest using caplog?

easy

A. with caplog.at_level(logging.WARNING):

B. with caplog.set_level(logging.WARNING):

C. caplog.at_level = logging.WARNING

D. caplog.capture_level(logging.WARNING)

Solution

1. Step 1: Identify correct method usage

   The correct method to set log level context is caplog.at_level() used with with statement.

2. Step 2: Check syntax correctness

   with caplog.at_level(logging.WARNING): uses with caplog.at_level(logging.WARNING): which is the documented and correct syntax.

3. Final Answer:

   with caplog.at_level(logging.WARNING): -> Option A

4. Quick Check:

   Use with caplog.at_level(level) for filtering [OK]

Hint: Use 'with caplog.at_level(level):' to filter logs [OK]

Common Mistakes:

• Using assignment instead of context manager
• Using non-existent methods like set_level or capture_level
• Missing the with statement

3. Given the code below, what log messages will be captured?

import logging

def test_logs(caplog):
    with caplog.at_level(logging.ERROR):
        logging.warning('Warning message')
        logging.error('Error message')
    return caplog.records

medium

A. Both 'Warning message' and 'Error message' are captured.

B. Only 'Warning message' is captured.

C. Only 'Error message' is captured.

D. No messages are captured.

Solution

1. Step 1: Understand log level filtering

   Setting caplog.at_level(logging.ERROR) captures logs at ERROR level or higher only.

2. Step 2: Analyze logged messages

   Warning is below ERROR level, so it is ignored. Error message is at ERROR level, so it is captured.

3. Final Answer:

   Only 'Error message' is captured. -> Option C

4. Quick Check:

   ERROR level filters out WARNING [OK]

Hint: Only logs at or above set level appear [OK]

Common Mistakes:

• Assuming WARNING logs appear when level is ERROR
• Thinking all logs are captured regardless of level
• Confusing ERROR and WARNING levels

4. Identify the error in the following pytest test code using caplog:

def test_example(caplog):
    caplog.at_level(logging.INFO)
    logging.debug('Debug message')
    logging.info('Info message')
    assert 'Info message' in caplog.text

medium

A. The assert statement syntax is incorrect.

B. caplog.at_level() must be used as a context manager with 'with'.

C. caplog.text does not exist; use caplog.records instead.

D. Logging messages must be uppercase strings.

Solution

1. Step 1: Check caplog.at_level usage

   The method at_level() must be used with a with statement to set the log level context.

2. Step 2: Identify the error

   Calling caplog.at_level(logging.INFO) without with does not set the level properly, so logs may not be filtered as expected.

3. Final Answer:

   caplog.at_level() must be used as a context manager with 'with'. -> Option B

4. Quick Check:

   Use 'with caplog.at_level(level):' always [OK]

Hint: Always use 'with' when calling caplog.at_level() [OK]

Common Mistakes:

• Calling at_level() without 'with' statement
• Assuming caplog.text is invalid (it exists)
• Thinking assert syntax is wrong

5. You want to test a function that logs messages at DEBUG, INFO, and ERROR levels. You only want to capture INFO and ERROR logs but exclude DEBUG. Which pytest code snippet correctly sets this up?

hard

A. with caplog.at_level(logging.ERROR):
func()
assert 'INFO message' in caplog.text

B. with caplog.at_level(logging.DEBUG):
func()
assert 'INFO message' in caplog.text

C. with caplog.at_level(logging.INFO):
func()
assert 'DEBUG message' in caplog.text

D. with caplog.at_level(logging.INFO):
func()
assert 'DEBUG message' not in caplog.text

Solution

1. Step 1: Determine log levels to capture

   Setting caplog.at_level(logging.INFO) captures INFO and ERROR logs but excludes DEBUG logs.

2. Step 2: Verify assertions

   with caplog.at_level(logging.INFO): func() assert 'DEBUG message' not in caplog.text asserts DEBUG messages are not in captured logs, which matches the requirement.

3. Final Answer:

   with caplog.at_level(logging.INFO): func() assert 'DEBUG message' not in caplog.text -> Option D

4. Quick Check:

   Set level INFO to exclude DEBUG logs [OK]

Hint: Set level to INFO to exclude DEBUG logs [OK]

Common Mistakes:

• Setting level too low and capturing DEBUG logs
• Setting level too high and missing INFO logs
• Wrong assertion about presence of DEBUG logs