Log level filtering in PyTest - Practice Problems & Coding Challenges

Challenge - 5 Problems

❓ Predict Output

intermediate

What is the output of this pytest log capture?

Given the following pytest test code with log level filtering, what will be captured in the test log output?

PyTest

import logging

def test_logging(caplog):
    with caplog.at_level(logging.WARNING):
        logging.debug('Debug message')
        logging.warning('Warning message')
        logging.error('Error message')
    print(caplog.text)

A Debug message\nWarning message\nError message\n

B Error message\n

C Warning message\nError message\n

D Debug message\n

💡 Hint

Remember that caplog.at_level sets the minimum log level to capture.

caplog.at_level(logging.WARNING) captures logs with level WARNING and above. DEBUG is below WARNING, so it is not captured.

❓ assertion

intermediate

Which assertion correctly checks that only ERROR logs are captured?

In a pytest test using caplog with level ERROR, which assertion correctly verifies that only ERROR messages are captured?

PyTest

import logging

def test_error_logs_only(caplog):
    with caplog.at_level(logging.ERROR):
        logging.warning('Warning message')
        logging.error('Error message')

A assert caplog.text == 'Warning message\nError message\n'

B assert 'Warning message' not in caplog.text and 'Error message' in caplog.text

C assert caplog.records[0].levelname == 'WARNING'

D assert 'Warning message' in caplog.text and 'Error message' in caplog.text

💡 Hint

Logs below ERROR level should not appear in caplog.text.

Since caplog is set to ERROR level, WARNING messages are not captured. The assertion must confirm WARNING is absent and ERROR is present.

🔧 Debug

advanced

Why does this pytest test fail to capture WARNING logs?

This pytest test intends to capture WARNING logs but fails. What is the cause?

PyTest

import logging

def test_logs(caplog):
    logging.warning('Warning message')
    with caplog.at_level(logging.WARNING):
        pass
    assert 'Warning message' in caplog.text

A The warning log is emitted before caplog.at_level context, so it is not captured.

B caplog.at_level does not capture WARNING logs by default.

C The assertion is incorrect; it should check caplog.records instead of caplog.text.

D Logging is disabled globally, so no logs are captured.

💡 Hint

Consider when the log message is emitted relative to the caplog context.

caplog.at_level only captures logs emitted inside its context. The warning is logged before entering the context, so it is missed.

❓ framework

advanced

How to configure pytest to capture only ERROR logs globally?

Which pytest configuration snippet sets the global log capture level to ERROR for all tests?

A

[pytest]
log_cli = true
log_cli_level = ERROR

B

[pytest]
log_cli = true
log_cli_level = WARNING

C

[pytest]
log_capture = true
log_level = ERROR

D

[pytest]
capture_logs = true
capture_level = ERROR

💡 Hint

Check pytest documentation for log_cli and log_cli_level options.

The correct pytest.ini options to enable live log capture at ERROR level are log_cli=true and log_cli_level=ERROR.

🧠 Conceptual

expert

What is the effect of setting caplog.set_level(logging.INFO) inside a test?

In pytest, what does calling caplog.set_level(logging.INFO) inside a test do?

A It only affects logs emitted before the call to set_level.

B It permanently changes the global logging level for all tests to INFO.

C It disables log capturing for the test.

D It changes the log capture level dynamically for the rest of the test, capturing INFO and above logs.

💡 Hint

Think about how caplog.set_level affects log capturing during test execution.

caplog.set_level changes the capture level dynamically during the test, affecting logs emitted after the call.

Practice

1. What does caplog.at_level() do in pytest?

easy

A. It sets the minimum log level to capture during a test.

B. It clears all captured logs before the test runs.

C. It disables logging for the entire test session.

D. It changes the log message format.

Solution

1. Step 1: Understand caplog usage

   caplog is a pytest fixture used to capture log messages during tests.

2. Step 2: Purpose of at_level()

   The method at_level() sets the minimum log level to capture, filtering out lower-level messages.

3. Final Answer:

   It sets the minimum log level to capture during a test. -> Option A

4. Quick Check:

   caplog.at_level() sets log capture level [OK]

Hint: Remember: at_level sets minimum log level to capture [OK]

Common Mistakes:

• Thinking it clears logs instead of setting level
• Confusing it with disabling logging
• Assuming it changes log format

2. Which of the following is the correct syntax to capture only WARNING and above logs in pytest using caplog?

easy

A. with caplog.at_level(logging.WARNING):

B. with caplog.set_level(logging.WARNING):

C. caplog.at_level = logging.WARNING

D. caplog.capture_level(logging.WARNING)

Solution

1. Step 1: Identify correct method usage

   The correct method to set log level context is caplog.at_level() used with with statement.

2. Step 2: Check syntax correctness

   with caplog.at_level(logging.WARNING): uses with caplog.at_level(logging.WARNING): which is the documented and correct syntax.

3. Final Answer:

   with caplog.at_level(logging.WARNING): -> Option A

4. Quick Check:

   Use with caplog.at_level(level) for filtering [OK]

Hint: Use 'with caplog.at_level(level):' to filter logs [OK]

Common Mistakes:

• Using assignment instead of context manager
• Using non-existent methods like set_level or capture_level
• Missing the with statement

3. Given the code below, what log messages will be captured?

import logging

def test_logs(caplog):
    with caplog.at_level(logging.ERROR):
        logging.warning('Warning message')
        logging.error('Error message')
    return caplog.records

medium

A. Both 'Warning message' and 'Error message' are captured.

B. Only 'Warning message' is captured.

C. Only 'Error message' is captured.

D. No messages are captured.

Solution

1. Step 1: Understand log level filtering

   Setting caplog.at_level(logging.ERROR) captures logs at ERROR level or higher only.

2. Step 2: Analyze logged messages

   Warning is below ERROR level, so it is ignored. Error message is at ERROR level, so it is captured.

3. Final Answer:

   Only 'Error message' is captured. -> Option C

4. Quick Check:

   ERROR level filters out WARNING [OK]

Hint: Only logs at or above set level appear [OK]

Common Mistakes:

• Assuming WARNING logs appear when level is ERROR
• Thinking all logs are captured regardless of level
• Confusing ERROR and WARNING levels

4. Identify the error in the following pytest test code using caplog:

def test_example(caplog):
    caplog.at_level(logging.INFO)
    logging.debug('Debug message')
    logging.info('Info message')
    assert 'Info message' in caplog.text

medium

A. The assert statement syntax is incorrect.

B. caplog.at_level() must be used as a context manager with 'with'.

C. caplog.text does not exist; use caplog.records instead.

D. Logging messages must be uppercase strings.

Solution

1. Step 1: Check caplog.at_level usage

   The method at_level() must be used with a with statement to set the log level context.

2. Step 2: Identify the error

   Calling caplog.at_level(logging.INFO) without with does not set the level properly, so logs may not be filtered as expected.

3. Final Answer:

   caplog.at_level() must be used as a context manager with 'with'. -> Option B

4. Quick Check:

   Use 'with caplog.at_level(level):' always [OK]

Hint: Always use 'with' when calling caplog.at_level() [OK]

Common Mistakes:

• Calling at_level() without 'with' statement
• Assuming caplog.text is invalid (it exists)
• Thinking assert syntax is wrong

5. You want to test a function that logs messages at DEBUG, INFO, and ERROR levels. You only want to capture INFO and ERROR logs but exclude DEBUG. Which pytest code snippet correctly sets this up?

hard

A. with caplog.at_level(logging.ERROR):
func()
assert 'INFO message' in caplog.text

B. with caplog.at_level(logging.DEBUG):
func()
assert 'INFO message' in caplog.text

C. with caplog.at_level(logging.INFO):
func()
assert 'DEBUG message' in caplog.text

D. with caplog.at_level(logging.INFO):
func()
assert 'DEBUG message' not in caplog.text

Solution

1. Step 1: Determine log levels to capture

   Setting caplog.at_level(logging.INFO) captures INFO and ERROR logs but excludes DEBUG logs.

2. Step 2: Verify assertions

   with caplog.at_level(logging.INFO): func() assert 'DEBUG message' not in caplog.text asserts DEBUG messages are not in captured logs, which matches the requirement.

3. Final Answer:

   with caplog.at_level(logging.INFO): func() assert 'DEBUG message' not in caplog.text -> Option D

4. Quick Check:

   Set level INFO to exclude DEBUG logs [OK]

Hint: Set level to INFO to exclude DEBUG logs [OK]

Common Mistakes:

• Setting level too low and capturing DEBUG logs
• Setting level too high and missing INFO logs
• Wrong assertion about presence of DEBUG logs