TestNG default reports in Selenium Java - Build an Automation Script

Verify TestNG default report generation after test execution

Preconditions (2)

Step 1: Run the TestNG test suite using the testng.xml or from the IDE

Step 2: Wait for the test execution to complete

Step 3: Locate the TestNG default report folder named 'test-output' in the project directory

Step 4: Open the 'index.html' file inside the 'test-output' folder in a web browser

Step 5: Verify that the report shows the test suite name, test results summary, and details of passed and failed tests

✅ Expected Result: TestNG default report is generated in 'test-output/index.html' showing correct test execution summary and details

Automation Requirements - TestNG with Selenium WebDriver

Assertions Needed:

Verify test method executed successfully

Verify test report folder 'test-output' is created

Verify 'index.html' file exists inside 'test-output'

Verify test result summary contains expected test method name and status

Best Practices:

Use TestNG annotations for setup and teardown

Use assertions from TestNG Assert class

Use explicit waits for Selenium actions if needed

Keep test methods independent

Run tests via TestNG suite XML or IDE runner

Automated Solution

Selenium Java

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

import java.io.File;

public class TestNGDefaultReportTest {
    WebDriver driver;

    @BeforeClass
    public void setup() {
        // Set path to chromedriver executable if needed
        // System.setProperty("webdriver.chrome.driver", "path/to/chromedriver");
        driver = new ChromeDriver();
    }

    @Test
    public void openGoogleAndCheckTitle() {
        driver.get("https://www.google.com");
        String title = driver.getTitle();
        Assert.assertTrue(title.contains("Google"), "Title should contain 'Google'");
    }

    @AfterClass
    public void teardown() {
        if (driver != null) {
            driver.quit();
        }
    }

    @Test
    public void verifyTestNGReportExists() {
        // This test verifies that TestNG default report is generated after tests run
        File reportFolder = new File("test-output");
        Assert.assertTrue(reportFolder.exists() && reportFolder.isDirectory(), "TestNG report folder 'test-output' should exist");

        File indexFile = new File(reportFolder, "index.html");
        Assert.assertTrue(indexFile.exists() && indexFile.isFile(), "TestNG report 'index.html' should exist inside 'test-output'");
    }
}

The code defines a TestNG test class with Selenium WebDriver.

setup() method initializes ChromeDriver before tests.

openGoogleAndCheckTitle() navigates to Google and asserts the page title contains 'Google'.

teardown() quits the browser after tests.

verifyTestNGReportExists() checks if the 'test-output' folder and 'index.html' report file exist after test execution.

This confirms that TestNG default reports are generated automatically after running tests.

Assertions use TestNG's Assert class to validate conditions.

Running this test class via TestNG will produce the default HTML report in 'test-output' folder.

Common Mistakes - 4 Pitfalls

Not quitting the WebDriver after tests

Checking for report files before test execution completes

{'mistake': 'Hardcoding absolute paths for report folder', 'why_bad': 'Makes tests less portable and harder to run on different machines', 'correct_approach': "Use relative paths like 'test-output' which TestNG creates in project root"}

Not using TestNG assertions for verification

Bonus Challenge

Now add data-driven testing with 3 different URLs and verify their page titles

Show Hint

Practice

1. What is the main purpose of TestNG default reports?

easy

A. To write custom test cases manually

B. To automatically show test results after running tests

C. To compile Java code before testing

D. To create database backups

Solution

1. Step 1: Understand what TestNG default reports do

   TestNG default reports automatically display the results of tests after execution without extra coding.

2. Step 2: Compare options with this function

   Only To automatically show test results after running tests matches this description; others describe unrelated tasks.

3. Final Answer:

   To automatically show test results after running tests -> Option B

4. Quick Check:

   TestNG default reports = automatic test result display [OK]

Hint: Default reports show results automatically after tests run [OK]

Common Mistakes:

• Thinking reports require extra code
• Confusing reports with code compilation
• Assuming reports manage databases

2. Which file is generated by TestNG by default to show test results?

easy

A. test-output/index.html

B. src/test/java/Test.java

C. pom.xml

D. log.txt

Solution

1. Step 1: Identify the default report file location

   TestNG generates reports inside the 'test-output' folder, with 'index.html' as the main report file.

2. Step 2: Check other options for correctness

   Other files are unrelated: source code, build config, or logs.

3. Final Answer:

   test-output/index.html -> Option A

4. Quick Check:

   Default report file = test-output/index.html [OK]

Hint: Look inside test-output folder for default reports [OK]

Common Mistakes:

• Confusing source code files with report files
• Expecting reports in pom.xml or logs
• Not checking the test-output directory

3. Given this TestNG test run, what will the default report show if one test fails and two pass?

medium

A. Passed: 2, Failed: 1, Skipped: 0

B. Passed: 3, Failed: 0, Skipped: 0

C. Passed: 1, Failed: 2, Skipped: 0

D. Passed: 0, Failed: 3, Skipped: 0

Solution

1. Step 1: Analyze the test results given

   There are two tests that passed and one test that failed.

2. Step 2: Match the counts to the report format

   The report shows counts as Passed: 2, Failed: 1, Skipped: 0.

3. Final Answer:

   Passed: 2, Failed: 1, Skipped: 0 -> Option A

4. Quick Check:

   Test counts in report = Passed 2, Failed 1 [OK]

Hint: Count tests passed and failed to match report summary [OK]

Common Mistakes:

• Mixing passed and failed counts
• Assuming skipped tests when none exist
• Misreading the test result numbers

4. You ran TestNG tests but the default report shows no tests executed. What is the likely cause?

medium

A. Java version is outdated

B. TestNG jar file is corrupted

C. Report folder is manually deleted

D. Test methods are missing the @Test annotation

Solution

1. Step 1: Understand why no tests run

   If test methods lack the @Test annotation, TestNG will not recognize or run them.

2. Step 2: Evaluate other options

   Corrupted jar or Java version issues usually cause errors, not silent no tests. Deleting report folder does not affect test execution.

3. Final Answer:

   Test methods are missing the @Test annotation -> Option D

4. Quick Check:

   No @Test annotation = no tests run [OK]

Hint: Check @Test annotation on test methods first [OK]

Common Mistakes:

• Assuming report folder deletion stops tests
• Blaming Java version without error messages
• Ignoring missing @Test annotations

5. How can you customize the TestNG default report to include screenshots on failure?

hard

A. By disabling default reports and writing logs to console only

B. By editing the test-output/index.html file manually after tests run

C. By adding listeners and embedding screenshots in the report via ITestListener

D. By renaming the test-output folder before running tests

Solution

1. Step 1: Understand default report customization

   TestNG default reports do not support screenshots by default; customization requires listeners like ITestListener.

2. Step 2: Evaluate options for adding screenshots

   Adding listeners to capture screenshots on failure and embed them is the correct approach. Manual editing or renaming folders won't add screenshots. Disabling reports removes them entirely.

3. Final Answer:

   By adding listeners and embedding screenshots in the report via ITestListener -> Option C

4. Quick Check:

   Use ITestListener to add screenshots [OK]

Hint: Use ITestListener to add screenshots on failure [OK]

Common Mistakes:

• Trying to edit report files manually
• Renaming folders expecting new features
• Disabling reports instead of customizing