@pytest.mark.xfail for expected failures - Build an Automation Script

Automate a test that is expected to fail using @pytest.mark.xfail

Preconditions (2)

Step 1: Create a test function that asserts 1 + 1 equals 3

Step 2: Mark the test function with @pytest.mark.xfail to indicate it is expected to fail

Step 3: Run the test suite using pytest

Step 4: Observe that the test is reported as an expected failure, not a failure

✅ Expected Result: The test is reported as xfailed (expected failure) by pytest, not as a failure or error

Automation Requirements - pytest

Assertions Needed:

Assert that 1 + 1 == 3 inside the test function

Best Practices:

Use @pytest.mark.xfail decorator to mark expected failures

Keep test functions simple and focused

Run tests with clear output to verify xfail behavior

Automated Solution

PyTest

import pytest

@pytest.mark.xfail
 def test_addition_expected_failure():
     assert 1 + 1 == 3

if __name__ == "__main__":
    import sys
    import pytest
    sys.exit(pytest.main([__file__]))

The @pytest.mark.xfail decorator marks the test as an expected failure. This means pytest will run the test but will not count it as a failure if it fails. Instead, it will report it as 'xfailed'.

The test function test_addition_expected_failure asserts that 1 + 1 equals 3, which is false, so it will fail normally. But because of the @pytest.mark.xfail, pytest expects this failure.

The if __name__ == "__main__" block allows running the test file directly, which helps beginners run and see the test output easily.

Common Mistakes - 3 Pitfalls

Not using @pytest.mark.xfail and expecting the test to pass

Using @pytest.mark.xfail but the test actually passes

Marking too many tests as xfail without fixing them later

Bonus Challenge

Now add three test functions with different expected failures using @pytest.mark.xfail, each failing for a different reason.

Show Hint

Practice

1. What is the main purpose of using @pytest.mark.xfail in a test?

easy

A. To mark tests that are expected to fail due to known issues

B. To skip tests that are not ready to run

C. To mark tests that should always pass

D. To run tests multiple times automatically

Solution

1. Step 1: Understand the purpose of @pytest.mark.xfail

   This decorator is used to mark tests that are expected to fail because of known bugs or unfinished features.

2. Step 2: Differentiate from other decorators

   Unlike skipping tests, xfail runs the test but expects failure. It does not mark tests to always pass or run multiple times.

3. Final Answer:

   To mark tests that are expected to fail due to known issues -> Option A

4. Quick Check:

   @pytest.mark.xfail = expected failure [OK]

Hint: xfail means test expected to fail, not skip or pass [OK]

Common Mistakes:

• Confusing xfail with skip
• Thinking xfail forces test to pass
• Assuming xfail reruns tests

2. Which of the following is the correct syntax to mark a test as expected to fail with strict checking in pytest?

easy

A. @pytest.mark.skip(strict=True)
def test_example(): pass

B. @pytest.mark.xfail(strict=True)
def test_example(): pass

C. @pytest.xfail(strict=True)
def test_example(): pass

D. @pytest.mark.xfail(strict=False)
def test_example(): pass

Solution

1. Step 1: Recall correct decorator usage

   The correct decorator is @pytest.mark.xfail with parentheses for parameters.

2. Step 2: Understand strict parameter

   Setting strict=True means pytest will fail the test if it unexpectedly passes.

3. Final Answer:

   @pytest.mark.xfail(strict=True)\ndef test_example(): pass -> Option B

4. Quick Check:

   Correct syntax uses @pytest.mark.xfail(strict=True) [OK]

Hint: Use @pytest.mark.xfail(strict=True) with parentheses [OK]

Common Mistakes:

• Using @pytest.xfail instead of @pytest.mark.xfail
• Confusing skip with xfail
• Omitting parentheses for parameters

3. Consider this pytest code snippet:

@pytest.mark.xfail(strict=True)
def test_divide():
    assert 1 / 0 == 0

What will be the test result when running pytest?

medium

A. Test passes because failure is expected

B. Test fails because division by zero raises an error

C. Test is skipped due to xfail

D. Test fails because strict=True causes failure on error

Solution

1. Step 1: Analyze the test code

   The test asserts 1 divided by 0 equals 0, which raises a ZeroDivisionError, causing failure.

2. Step 2: Understand xfail with strict=True

   Since failure is expected, the test is marked as xfail and will pass the test suite. strict=True only fails if the test unexpectedly passes.

3. Final Answer:

   Test passes because failure is expected -> Option A

4. Quick Check:

   xfail with error = pass test [OK]

Hint: xfail passes if test fails; strict=True fails only if test passes [OK]

Common Mistakes:

• Thinking strict=True fails on error
• Confusing xfail with skip
• Expecting test to fail pytest run

4. You wrote this test:

@pytest.mark.xfail(strict=True)
def test_sum():
    assert sum([1, 2, 3]) == 5

When you run pytest, the test fails. What is the most likely reason?

medium

A. The test unexpectedly passed, causing failure due to strict=True

B. The xfail decorator was not applied correctly

C. The test failed because the assertion is incorrect

D. The test was skipped accidentally

Solution

1. Step 1: Check the assertion logic

   The sum of [1, 2, 3] is 6, but the test asserts it equals 5, so the assertion fails.

2. Step 2: Understand strict=True effect

   strict=True causes failure only if the test unexpectedly passes, but here the test fails as expected.

3. Final Answer:

   The test failed because the assertion is incorrect -> Option C

4. Quick Check:

   Wrong assertion causes failure, not strict=True [OK]

Hint: Check assertion values carefully; strict=True fails only on unexpected pass [OK]

Common Mistakes:

• Assuming strict=True causes failure on test fail
• Thinking test was skipped
• Believing decorator syntax is wrong

5. You have a test that sometimes passes and sometimes fails due to a flaky external API. You want pytest to mark it as expected failure but fail the test run if it unexpectedly passes. Which is the best way to write this?

hard

A. @pytest.mark.xfail()
def test_api():
assert call_api() == 'OK'

B. @pytest.mark.skip(reason='Flaky API')
def test_api():
assert call_api() == 'OK'

C. @pytest.mark.xfail(strict=False, reason='Flaky API')
def test_api():
assert call_api() == 'OK'

D. @pytest.mark.xfail(strict=True, reason='Flaky API')
def test_api():
assert call_api() == 'OK'

Solution

1. Step 1: Identify requirement for strict checking

   You want pytest to fail the test run if the test unexpectedly passes, so strict=True is needed.

2. Step 2: Use xfail with reason for clarity

   Marking the test with @pytest.mark.xfail(strict=True, reason='Flaky API') clearly documents the expected failure and enforces strict checking.

3. Final Answer:

   @pytest.mark.xfail(strict=True, reason='Flaky API')\ndef test_api():\n assert call_api() == 'OK' -> Option D

4. Quick Check:

   xfail with strict=True and reason = best practice [OK]

Hint: Use strict=True to catch unexpected passes on flaky tests [OK]

Common Mistakes:

• Using skip instead of xfail
• Omitting strict=True when needed
• Not providing reason for clarity