Retry analyzer for failures in Selenium Java

Introduction

Sometimes tests fail because of temporary problems. Retry analyzer helps run the test again automatically to check if it passes on retry.

When a test fails due to slow loading of a web page.

When a test fails because of a temporary network glitch.

When a test intermittently fails due to timing issues.

When you want to reduce false test failures caused by flaky tests.

Syntax

Selenium Java

public class RetryAnalyzer implements IRetryAnalyzer {
    private int count = 0;
    private static final int maxTry = 3;

    @Override
    public boolean retry(ITestResult result) {
        if (count < maxTry) {
            count++;
            return true;
        }
        return false;
    }
}

This class implements the IRetryAnalyzer interface from TestNG.

The retry method returns true to retry the test, false to stop retrying.

Examples

This retry analyzer retries a failed test up to 2 times.

Selenium Java

public class RetryAnalyzer implements IRetryAnalyzer {
    private int count = 0;
    private static final int maxTry = 2;

    @Override
    public boolean retry(ITestResult result) {
        if (count < maxTry) {
            count++;
            return true;
        }
        return false;
    }
}

This test uses the retry analyzer to rerun if it fails.

Selenium Java

@Test(retryAnalyzer = RetryAnalyzer.class)
public void testExample() {
    Assert.assertTrue(new java.util.Random().nextBoolean());
}

Sample Program

This test sometimes fails randomly. The retry analyzer will retry it up to 2 times if it fails.

Selenium Java

import org.testng.Assert;
import org.testng.ITestResult;
import org.testng.annotations.Test;
import org.testng.IRetryAnalyzer;

public class RetryAnalyzer implements IRetryAnalyzer {
    private int count = 0;
    private static final int maxTry = 2;

    @Override
    public boolean retry(ITestResult result) {
        if (count < maxTry) {
            count++;
            System.out.println("Retrying test " + result.getName() + " for " + count + " time(s)");
            return true;
        }
        return false;
    }
}

public class SampleTest {

    @Test(retryAnalyzer = RetryAnalyzer.class)
    public void flakyTest() {
        boolean pass = Math.random() > 0.7; // 30% chance to fail
        System.out.println("Test run, pass? " + pass);
        Assert.assertTrue(pass, "Random failure");
    }
}

Important Notes

Retry analyzer only retries failed tests, not passed ones.

Use retry analyzer carefully to avoid hiding real bugs.

Print statements help see retry attempts in console output.

Summary

Retry analyzer helps rerun failed tests automatically.

It reduces false failures caused by temporary issues.

Implemented by overriding retry method from IRetryAnalyzer.