Why @pytest.mark.xfail for expected failures? - Purpose & Use Cases

The Big Idea

What if your test failures could tell you which ones really need your attention?

The Scenario

Imagine you have a test that checks a feature still under development or a known bug that is not fixed yet.

You run all tests manually and see this test fail every time, mixing it with unexpected failures.

The Problem

Manually tracking which failures are expected wastes time and causes confusion.

You might spend hours investigating failures that you already know about.

This slows down your work and increases frustration.

The Solution

The @pytest.mark.xfail tag tells pytest that a test is expected to fail.

Pytest then marks these failures differently, so you can focus on new, unexpected problems.

This saves time and keeps your test reports clear.

Before vs After

✗ Before

def test_feature():
    assert feature() == expected  # fails but no special mark

✓ After

@pytest.mark.xfail
def test_feature():
    assert feature() == expected

What It Enables

You can easily separate known issues from new bugs, making your testing smarter and faster.

Real Life Example

When a new feature is half-built, you mark its test with @pytest.mark.xfail so your test suite stays green while you finish the work.

Key Takeaways

Manually tracking expected failures wastes time and causes confusion.

@pytest.mark.xfail clearly marks tests expected to fail.

This helps focus on real new problems and speeds up testing.

Practice

1. What is the main purpose of using @pytest.mark.xfail in a test?

easy

A. To mark tests that are expected to fail due to known issues

B. To skip tests that are not ready to run

C. To mark tests that should always pass

D. To run tests multiple times automatically

Solution

1. Step 1: Understand the purpose of @pytest.mark.xfail

   This decorator is used to mark tests that are expected to fail because of known bugs or unfinished features.

2. Step 2: Differentiate from other decorators

   Unlike skipping tests, xfail runs the test but expects failure. It does not mark tests to always pass or run multiple times.

3. Final Answer:

   To mark tests that are expected to fail due to known issues -> Option A

4. Quick Check:

   @pytest.mark.xfail = expected failure [OK]

Hint: xfail means test expected to fail, not skip or pass [OK]

Common Mistakes:

• Confusing xfail with skip
• Thinking xfail forces test to pass
• Assuming xfail reruns tests

2. Which of the following is the correct syntax to mark a test as expected to fail with strict checking in pytest?

easy

A. @pytest.mark.skip(strict=True)
def test_example(): pass

B. @pytest.mark.xfail(strict=True)
def test_example(): pass

C. @pytest.xfail(strict=True)
def test_example(): pass

D. @pytest.mark.xfail(strict=False)
def test_example(): pass

Solution

1. Step 1: Recall correct decorator usage

   The correct decorator is @pytest.mark.xfail with parentheses for parameters.

2. Step 2: Understand strict parameter

   Setting strict=True means pytest will fail the test if it unexpectedly passes.

3. Final Answer:

   @pytest.mark.xfail(strict=True)\ndef test_example(): pass -> Option B

4. Quick Check:

   Correct syntax uses @pytest.mark.xfail(strict=True) [OK]

Hint: Use @pytest.mark.xfail(strict=True) with parentheses [OK]

Common Mistakes:

• Using @pytest.xfail instead of @pytest.mark.xfail
• Confusing skip with xfail
• Omitting parentheses for parameters

3. Consider this pytest code snippet:

@pytest.mark.xfail(strict=True)
def test_divide():
    assert 1 / 0 == 0

What will be the test result when running pytest?

medium

A. Test passes because failure is expected

B. Test fails because division by zero raises an error

C. Test is skipped due to xfail

D. Test fails because strict=True causes failure on error

Solution

1. Step 1: Analyze the test code

   The test asserts 1 divided by 0 equals 0, which raises a ZeroDivisionError, causing failure.

2. Step 2: Understand xfail with strict=True

   Since failure is expected, the test is marked as xfail and will pass the test suite. strict=True only fails if the test unexpectedly passes.

3. Final Answer:

   Test passes because failure is expected -> Option A

4. Quick Check:

   xfail with error = pass test [OK]

Hint: xfail passes if test fails; strict=True fails only if test passes [OK]

Common Mistakes:

• Thinking strict=True fails on error
• Confusing xfail with skip
• Expecting test to fail pytest run

4. You wrote this test:

@pytest.mark.xfail(strict=True)
def test_sum():
    assert sum([1, 2, 3]) == 5

When you run pytest, the test fails. What is the most likely reason?

medium

A. The test unexpectedly passed, causing failure due to strict=True

B. The xfail decorator was not applied correctly

C. The test failed because the assertion is incorrect

D. The test was skipped accidentally

Solution

1. Step 1: Check the assertion logic

   The sum of [1, 2, 3] is 6, but the test asserts it equals 5, so the assertion fails.

2. Step 2: Understand strict=True effect

   strict=True causes failure only if the test unexpectedly passes, but here the test fails as expected.

3. Final Answer:

   The test failed because the assertion is incorrect -> Option C

4. Quick Check:

   Wrong assertion causes failure, not strict=True [OK]

Hint: Check assertion values carefully; strict=True fails only on unexpected pass [OK]

Common Mistakes:

• Assuming strict=True causes failure on test fail
• Thinking test was skipped
• Believing decorator syntax is wrong

5. You have a test that sometimes passes and sometimes fails due to a flaky external API. You want pytest to mark it as expected failure but fail the test run if it unexpectedly passes. Which is the best way to write this?

hard

A. @pytest.mark.xfail()
def test_api():
assert call_api() == 'OK'

B. @pytest.mark.skip(reason='Flaky API')
def test_api():
assert call_api() == 'OK'

C. @pytest.mark.xfail(strict=False, reason='Flaky API')
def test_api():
assert call_api() == 'OK'

D. @pytest.mark.xfail(strict=True, reason='Flaky API')
def test_api():
assert call_api() == 'OK'

Solution

1. Step 1: Identify requirement for strict checking

   You want pytest to fail the test run if the test unexpectedly passes, so strict=True is needed.

2. Step 2: Use xfail with reason for clarity

   Marking the test with @pytest.mark.xfail(strict=True, reason='Flaky API') clearly documents the expected failure and enforces strict checking.

3. Final Answer:

   @pytest.mark.xfail(strict=True, reason='Flaky API')\ndef test_api():\n assert call_api() == 'OK' -> Option D

4. Quick Check:

   xfail with strict=True and reason = best practice [OK]

Hint: Use strict=True to catch unexpected passes on flaky tests [OK]

Common Mistakes:

• Using skip instead of xfail
• Omitting strict=True when needed
• Not providing reason for clarity