Report publishing in CI in Selenium Java - Build an Automation Script

Verify test report is generated and published in CI pipeline

Preconditions (3)

Step 1: Trigger the CI pipeline to run the Selenium test suite

Step 2: Wait for the test execution to complete

Step 3: Locate the generated test report file (e.g., target/surefire-reports/index.html)

Step 4: Verify the report file exists and is not empty

Step 5: Check the CI pipeline logs or artifacts section to confirm the report is published and accessible

✅ Expected Result: The Selenium test suite runs successfully in CI, the test report file is generated and published as a pipeline artifact, and it can be accessed from the CI interface.

Automation Requirements - Selenium with TestNG and Maven

Assertions Needed:

Assert test report file exists after test execution

Assert test report file size is greater than zero

Best Practices:

Use TestNG annotations for setup and teardown

Use Maven Surefire plugin for report generation

Use explicit waits in Selenium tests

Keep test code and report generation separate

Ensure reports are archived as CI artifacts

Automated Solution

Selenium Java

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import java.io.File;
import java.time.Duration;

public class ReportPublishingTest {
    private WebDriver driver;

    @BeforeClass
    public void setUp() {
        // Set path to chromedriver executable if needed
        // System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver");
        driver = new ChromeDriver();
        driver.manage().timeouts().implicitlyWait(Duration.ofSeconds(10));
    }

    @Test
    public void sampleTest() {
        driver.get("https://example.com");
        String title = driver.getTitle();
        Assert.assertTrue(title.contains("Example Domain"), "Title should contain 'Example Domain'");
    }

    @AfterClass
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }

    // This method simulates verification of report file after tests run
    @Test(dependsOnMethods = {"sampleTest"})
    public void verifyReportFile() {
        // Path to Maven Surefire report file
        File reportFile = new File("target/surefire-reports/index.html");
        Assert.assertTrue(reportFile.exists(), "Test report file should exist");
        Assert.assertTrue(reportFile.length() > 0, "Test report file should not be empty");
    }
}

The setUp method initializes the Chrome WebDriver with an implicit wait to handle element loading.

The sampleTest method navigates to example.com and asserts the page title contains 'Example Domain' to simulate a passing test.

The tearDown method closes the browser after tests complete.

The verifyReportFile test runs after the sample test and checks if the Maven Surefire report file target/surefire-reports/index.html exists and is not empty, ensuring the report was generated.

In a real CI environment, the pipeline should archive this report file as an artifact for publishing. This test confirms the report generation step.

Common Mistakes - 4 Pitfalls

Not waiting for tests to complete before checking report file

Hardcoding absolute paths for report files

Not archiving or publishing the report as a CI artifact

Using Thread.sleep instead of implicit or explicit waits in Selenium tests

Bonus Challenge

Now add data-driven testing with 3 different URLs and verify their titles

Show Hint