Asserting log messages in PyTest

Introduction

We check log messages to make sure our program reports important events correctly. This helps us find problems and understand what happened during a test.

When you want to confirm that a warning or error message is shown during a test.

When your code logs important steps and you want to verify those steps happened.

When debugging and you want to ensure specific log messages appear for certain actions.

When testing that sensitive information is not logged by mistake.

When you want to check that your logging setup works as expected.

Syntax

PyTest

with caplog.at_level(logging.LEVEL):
    # code that triggers logging
    assert 'expected message' in caplog.text

caplog is a pytest fixture that captures log messages during tests.

You can set the log level with caplog.at_level() to capture messages of that level or higher.

Examples

This test checks that a warning message is logged.

PyTest

def test_warning_log(caplog):
    import logging
    with caplog.at_level(logging.WARNING):
        logging.warning('This is a warning')
    assert 'This is a warning' in caplog.text

This test asserts the error message appears in the captured logs.

PyTest

def test_error_log(caplog):
    import logging
    with caplog.at_level(logging.ERROR):
        logging.error('Error happened')
    assert any('Error happened' in msg for msg in caplog.messages)

Sample Program

This test runs a function that logs messages at different levels. It then checks that all these messages appear in the captured logs.

PyTest

import logging
import pytest

def function_that_logs():
    logging.info('Starting process')
    logging.warning('A warning occurred')
    logging.error('An error occurred')

def test_logs(caplog):
    with caplog.at_level(logging.INFO):
        function_that_logs()
    assert 'Starting process' in caplog.text
    assert 'A warning occurred' in caplog.text
    assert 'An error occurred' in caplog.text

if __name__ == '__main__':
    pytest.main([__file__])

Important Notes

Always import logging to use log levels and log messages.

Use caplog.text to get all captured logs as one string, or caplog.records for detailed log records.

Set the log level carefully to capture the messages you want.

Summary

Use caplog fixture to capture logs in pytest tests.

Check log messages with assertions on caplog.text or caplog.messages.

Set log level with caplog.at_level() to control which messages are captured.

Practice

1. What is the purpose of the caplog fixture in pytest when testing log messages?

easy

A. To capture log messages generated during test execution

B. To mock external API calls

C. To measure test execution time

D. To generate random test data

Solution

1. Step 1: Understand the role of caplog

   The caplog fixture is designed to capture log messages during pytest test runs.

2. Step 2: Identify correct usage context

   It helps verify that expected log messages are produced, which is essential for testing logging behavior.

3. Final Answer:

   To capture log messages generated during test execution -> Option A

4. Quick Check:

   caplog captures logs [OK]

Hint: Remember: caplog captures logs during tests [OK]

Common Mistakes:

• Confusing caplog with mocking tools
• Thinking caplog measures time
• Assuming caplog generates data

2. Which of the following is the correct way to set the log level to capture WARNING messages but not INFO messages using caplog in pytest?

easy

A. caplog.set_level(logging.INFO)

B. caplog.set_level(logging.ERROR)

C. caplog.set_level(logging.DEBUG)

D. caplog.set_level(logging.WARNING)

Solution

1. Step 1: Understand log levels

   To capture WARNING messages but not INFO, the log level must be set to WARNING.

2. Step 2: Identify correct method call

   Using caplog.set_level(logging.WARNING) sets the capture level to WARNING, capturing WARNING and above.

3. Final Answer:

   caplog.set_level(logging.WARNING) -> Option D

4. Quick Check:

   Set level to WARNING to capture warnings [OK]

Hint: Set caplog level to desired severity (e.g., WARNING) [OK]

Common Mistakes:

• Setting level too high to capture lower severity logs
• Using incorrect logging level constants
• Not calling set_level at all

3. Given the following pytest test code, what will be the result of the assertion?

import logging

def test_log_message(caplog):
    with caplog.at_level(logging.ERROR):
        logging.error('Error occurred')
    assert 'Error occurred' in caplog.text

medium

A. Test passes because the error message is captured

B. Test fails because caplog.text is empty

C. Test fails due to syntax error

D. Test passes but assertion is incorrect

Solution

1. Step 1: Analyze log capture level

   The test sets caplog to capture ERROR level logs, and an error message is logged.

2. Step 2: Check assertion logic

   The assertion checks if the logged message is in caplog.text, which contains all captured logs as text.

3. Final Answer:

   Test passes because the error message is captured -> Option A

4. Quick Check:

   Logged ERROR message found in caplog.text [OK]

Hint: Check caplog.text contains your logged message [OK]

Common Mistakes:

• Assuming caplog.text is empty without setting level
• Confusing caplog.records with caplog.text
• Forgetting to set log level

4. Identify the error in the following pytest test code that uses caplog to assert a log message:

import logging

def test_info_log(caplog):
    logging.info('Info issued')
    assert 'Info issued' in caplog.text

medium

A. Logging function used is wrong

B. Incorrect assertion syntax

C. Missing setting log level to capture INFO messages

D. caplog fixture is not imported

Solution

1. Step 1: Check log level setting

   The test logs an INFO but does not set caplog to capture INFO level logs, so no logs are captured.

2. Step 2: Understand impact on assertion

   Because no logs are captured, caplog.text is empty and the assertion fails.

3. Final Answer:

   Missing setting log level to capture INFO messages -> Option C

4. Quick Check:

   Set caplog level to INFO before logging [OK]

Hint: Always set caplog level before logging messages [OK]

Common Mistakes:

• Not using caplog.at_level or caplog.set_level
• Assuming logs are captured by default
• Incorrectly placing assertion before logging

5. You want to test that your function logs an INFO message containing the word 'success'. Which pytest caplog assertion correctly verifies this?

hard

A. assert caplog.records.count('success') == 1

B. assert any('success' in record.message for record in caplog.records if record.levelname == 'INFO')

C. assert caplog.text == 'success'

D. assert 'success' in caplog.records[0]

Solution

1. Step 1: Understand caplog.records structure

   caplog.records is a list of log record objects with attributes like message and levelname.

2. Step 2: Choose correct assertion logic

   assert any('success' in record.message for record in caplog.records if record.levelname == 'INFO') checks if any INFO level record contains 'success' in its message, which correctly verifies the log content.

3. Final Answer:

   assert any('success' in record.message for record in caplog.records if record.levelname == 'INFO') -> Option B

4. Quick Check:

   Use caplog.records with message and levelname for assertions [OK]

Hint: Use caplog.records with message and levelname for checks [OK]

Common Mistakes:

• Using count on records list incorrectly
• Comparing caplog.text directly to a string
• Accessing caplog.records elements as strings