Flaky test detection and retry in PyTest - Test Execution Trace

Test Overview

This test checks a function that sometimes fails randomly. It uses retry to run the test again if it fails, to detect flaky behavior and pass if any retry succeeds.

Test Code - pytest

PyTest

import random
import pytest

@pytest.mark.flaky(reruns=2, reruns_delay=1)
def test_random_flaky_behavior():
    value = random.choice([True, False])
    assert value is True, "Random failure to simulate flakiness"

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Test starts	Pytest test runner initializes and prepares to run test_random_flaky_behavior	-	PASS
2	Test executes first run: random.choice returns False	Test function runs, random value is False	assert value is True	FAIL
3	Pytest detects failure and triggers retry after 1 second delay	Waiting 1 second before retrying test	-
4	Test executes second run: random.choice returns True	Test function runs, random value is True	assert value is True	PASS
5	Test passes due to successful retry	Test marked as PASSED in test report	-	PASS

Failure Scenario

Failing Condition: All retries fail with random value False

Execution Trace Quiz - 3 Questions

What causes the test to retry in this example?

A The test passed on the first run

B The test was skipped

C The test failed an assertion on the first run

D The test timed out

Key Result

Using retries helps detect flaky tests by allowing them to pass if they succeed on a retry, reducing false failures in test reports.