Flaky test detection in JUnit - Test Execution Trace

Test Overview

This test runs a simple method multiple times to detect if it sometimes fails unpredictably, which indicates a flaky test.

Test Code - JUnit 5

JUnit

import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.Assertions;
import java.util.Random;

public class FlakyTestDetection {

    private boolean sometimesFails() {
        // Simulate flaky behavior: fails randomly 30% of the time
        return new Random().nextInt(10) >= 7;
    }

    @RepeatedTest(10)
    void testSometimesFails() {
        Assertions.assertTrue(sometimesFails(), "Method failed unexpectedly");
    }
}

Execution Trace - 7 Steps

Step	Action	System State	Assertion	Result
1	Test runner starts and prepares to run testSometimesFails 10 times	JUnit test environment initialized	-	PASS
2	Runs testSometimesFails iteration 1: calls sometimesFails()	sometimesFails returns true (success)	assertTrue passes	PASS
3	Runs testSometimesFails iteration 2: calls sometimesFails()	sometimesFails returns true (success)	assertTrue passes	PASS
4	Runs testSometimesFails iteration 3: calls sometimesFails()	sometimesFails returns false (failure)	assertTrue fails with message 'Method failed unexpectedly'	FAIL
5	Test runner records failure for iteration 3	Test report marks iteration 3 as failed	-	PASS
6	Runs testSometimesFails iteration 4 to 10: calls sometimesFails()	sometimesFails returns true for most, possibly false for some	assertTrue passes or fails accordingly	PASS OR FAIL DEPENDING ON ITERATION
7	Test runner finishes all 10 iterations and aggregates results	Test report shows some iterations passed, some failed	Overall test marked as flaky due to intermittent failures	FAIL

Failure Scenario

Failing Condition: sometimesFails() returns false randomly causing assertion failure

Execution Trace Quiz - 3 Questions

What does the @RepeatedTest(10) annotation do in this test?

A Runs 10 different test methods once each

B Runs the test method 10 times to detect flaky behavior

C Runs the test method once with 10 assertions

D Runs the test method 10 times in parallel threads

Key Result

Running tests multiple times helps detect flaky tests that fail unpredictably, allowing you to improve test reliability by stabilizing the code or environment.