Flaky test detection in JUnit - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is a flaky test?

A flaky test is a test that sometimes passes and sometimes fails without any changes in the code. It behaves unpredictably, making it hard to trust test results.

beginner

Why are flaky tests problematic in software testing?

Flaky tests cause confusion because they give inconsistent results. They can hide real bugs or waste time investigating false failures.

beginner

Name one common cause of flaky tests.

One common cause is timing issues, like tests depending on slow network responses or asynchronous operations that are not properly handled.

intermediate

How can you detect flaky tests using JUnit?

You can run the same test multiple times in JUnit and check if it sometimes fails and sometimes passes. Tools or plugins can help automate this detection.

intermediate

What is a simple JUnit code example to run a test multiple times to detect flakiness?

Use @RepeatedTest annotation to run a test multiple times. If results vary, the test might be flaky.
Example:

import org.junit.jupiter.api.RepeatedTest;
import static org.junit.jupiter.api.Assertions.assertTrue;

class FlakyTestExample {
  @RepeatedTest(5)
  void testSometimesFails() {
    boolean condition = Math.random() > 0.2; // 80% chance to pass
    assertTrue(condition, "Test failed randomly");
  }
}

What does a flaky test do?

A Always fails

B Always passes

C Runs faster than other tests

D Passes and fails unpredictably

A flaky test sometimes passes and sometimes fails without code changes.

Which of these is a common cause of flaky tests?

A Timing or race conditions

B Clear assertions

C Properly isolated tests

D Consistent test data

Timing or race conditions often cause flaky tests.

How can you detect flaky tests in JUnit?

A Run tests multiple times and check for inconsistent results

B Run tests once and trust results

C Ignore test failures

D Only run tests manually

Running tests multiple times helps find flaky tests by spotting inconsistent outcomes.

What JUnit annotation helps run a test multiple times?

A @Test

B @BeforeEach

C @RepeatedTest

D @AfterAll

@RepeatedTest runs a test multiple times to check for flakiness.

Why should flaky tests be fixed?

A They slow down the computer

B They cause unreliable test results

C They make tests run faster

D They increase code coverage

Flaky tests cause unreliable results, making it hard to trust testing.

Explain what a flaky test is and why it is important to detect it.

Think about tests that sometimes fail without code changes.

Describe a simple method to detect flaky tests using JUnit.

How can repeating a test help find flakiness?