TestWatcher for reporting in JUnit - Build an Automation Script

Automate test reporting using JUnit TestWatcher

Preconditions (2)

Step 1: Create a TestWatcher extension class that overrides testSuccessful, testFailed, and testDisabled methods

Step 2: In each overridden method, log the test method name and its result (success, failure, or disabled)

Step 3: Attach the TestWatcher extension to the sample test class using @ExtendWith annotation

Step 4: Run the test class

Step 5: Observe the console output for correct reporting of each test method's result

✅ Expected Result: The console shows messages indicating each test method's name and whether it passed, failed, or was skipped

Automation Requirements - JUnit 5

Assertions Needed:

Verify that the TestWatcher logs the correct test method name and status for each test

Best Practices:

Use @ExtendWith to register the TestWatcher extension

Use descriptive log messages for clarity

Keep the TestWatcher class reusable for other test classes

Avoid hardcoding test method names; use the provided ExtensionContext

Automated Solution

JUnit

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.TestWatcher;
import java.util.Optional;

class ReportingTestWatcher implements TestWatcher {

    @Override
    public void testSuccessful(ExtensionContext context) {
        String testName = context.getDisplayName();
        System.out.println("[PASS] Test succeeded: " + testName);
    }

    @Override
    public void testFailed(ExtensionContext context, Throwable cause) {
        String testName = context.getDisplayName();
        System.out.println("[FAIL] Test failed: " + testName + ". Reason: " + cause.getMessage());
    }

    @Override
    public void testDisabled(ExtensionContext context, Optional<String> reason) {
        String testName = context.getDisplayName();
        System.out.println("[SKIP] Test disabled: " + testName + ". Reason: " + reason.orElse("No reason provided"));
    }
}

@ExtendWith(ReportingTestWatcher.class)
public class SampleTests {

    @Test
    void testSuccess() {
        // This test will pass
        int sum = 2 + 3;
        assert sum == 5;
    }

    @Test
    void testFailure() {
        // This test will fail
        int product = 2 * 2;
        assert product == 5 : "Product should be 5";
    }

    @Test
    void testAnotherSuccess() {
        // Another passing test
        String text = "hello".toUpperCase();
        assert text.equals("HELLO");
    }

    @Test
    @org.junit.jupiter.api.Disabled("Not implemented yet")
    void testDisabled() {
        // This test is disabled
    }
}

The ReportingTestWatcher class implements JUnit 5's TestWatcher interface to listen to test lifecycle events.

We override testSuccessful, testFailed, and testDisabled methods to print messages to the console with the test name and status.

The SampleTests class uses @ExtendWith to attach the watcher. It contains tests that pass, fail, and one disabled test to demonstrate all cases.

When running the tests, the console will show clear messages indicating which tests passed, failed, or were skipped, helping with reporting.

Common Mistakes - 4 Pitfalls

Not using @ExtendWith to register the TestWatcher

{'mistake': 'Hardcoding test method names instead of using ExtensionContext', 'why_bad': 'Hardcoding names makes the watcher inflexible and error-prone if tests change.', 'correct_approach': "Use context.getDisplayName() to get the current test's name dynamically."}

Printing messages without clear status labels

Ignoring the cause parameter in testFailed method

Bonus Challenge

Extend the TestWatcher to write the test results to a file instead of console output.

Show Hint

Practice

1. What is the primary purpose of using TestWatcher in JUnit tests?

easy

A. To react to test lifecycle events like start, success, failure, and finish

B. To write assertions inside test methods

C. To create mock objects for testing

D. To configure database connections before tests

Solution

1. Step 1: Understand TestWatcher role

   TestWatcher is a JUnit class designed to listen to test lifecycle events such as when a test starts, succeeds, fails, or finishes.

2. Step 2: Differentiate from other test tasks

   Writing assertions, mocking, or configuring databases are separate concerns and not the main purpose of TestWatcher.

3. Final Answer:

   To react to test lifecycle events like start, success, failure, and finish -> Option A

4. Quick Check:

   TestWatcher = react to test events [OK]

Hint: TestWatcher listens to test events, not assertions or mocks [OK]

Common Mistakes:

• Confusing TestWatcher with assertion writing
• Thinking TestWatcher creates mocks
• Assuming TestWatcher manages test setup like DB connections

2. Which method signature correctly overrides the TestWatcher method that runs when a test fails?

easy

A. public void failed(ExtensionContext context, Throwable cause)

B. protected void failed(Throwable e, Description description)

C. void failed(Description description, Throwable e)

D. public void testFailed(ExtensionContext context, Throwable cause)

Solution

1. Step 1: Recall TestWatcher method signature

   The failed method in TestWatcher is public void failed(ExtensionContext context, Throwable cause).

2. Step 2: Check method modifiers and parameter order

   It must be public, return void, and parameters are ExtensionContext first, then Throwable.

3. Final Answer:

   public void failed(ExtensionContext context, Throwable cause) -> Option A

4. Quick Check:

   failed method = public void failed(ExtensionContext, Throwable) [OK]

Hint: Check parameter order and access modifier for overrides [OK]

Common Mistakes:

• Swapping parameter order
• Using protected instead of public
• Missing return type or wrong method name

3. Given the following TestWatcher subclass, what will be printed if the test exampleTest fails?

class MyWatcher extends TestWatcher {
  @Override
  public void failed(ExtensionContext context, Throwable cause) {
    context.getTestMethod().ifPresent(method -> System.out.println(method.getName() + " failed"));
  }
}

@Test
@ExtendWith(MyWatcher.class)
void exampleTest() {
  assertEquals(1, 2);
}

medium

A. No output printed

B. failed

C. exampleTest

D. exampleTest failed

Solution

1. Step 1: Understand what failed() prints

   The overridden failed method prints the test method name plus " failed".

2. Step 2: Identify test method name on failure

   The test method is named exampleTest, so the output will be "exampleTest failed".

3. Final Answer:

   exampleTest failed -> Option D

4. Quick Check:

   Print = methodName + " failed" = exampleTest failed [OK]

Hint: Print uses description.getMethodName() on failure [OK]

Common Mistakes:

• Assuming no output on failure
• Printing only 'failed' without method name
• Confusing method name with test class name

4. Identify the error in this TestWatcher subclass that aims to log test success:

class SuccessWatcher extends TestWatcher {
  @Override
  public void succeeded(Description description) {
    System.out.println(description.getDisplayName() + " succeeded");
  }
}

medium

A. Method signature is incorrect; it should include Throwable parameter

B. Method signature is incorrect; it should be public void succeeded(ExtensionContext context)

C. Method should be protected, not public

D. Method signature is incorrect; it should be public void succeeded(Description description, Throwable e)

Solution

1. Step 1: Check JUnit version and method signature

   In JUnit 5, TestWatcher methods use ExtensionContext parameter, not Description.

2. Step 2: Identify correct succeeded() signature

   The correct override is public void succeeded(ExtensionContext context) in JUnit 5.

3. Final Answer:

   Method signature is incorrect; it should be public void succeeded(ExtensionContext context) -> Option B

4. Quick Check:

   JUnit 5 TestWatcher uses ExtensionContext parameter [OK]

Hint: JUnit 5 TestWatcher methods use ExtensionContext, not Description [OK]

Common Mistakes:

• Using JUnit 4 signatures in JUnit 5
• Wrong parameter types or order
• Confusing access modifiers

5. You want to create a custom TestWatcher that logs test start and failure events, but only for tests whose names start with "critical". Which approach correctly implements this behavior in JUnit 5?

hard

A. Override starting(Description description) and failed(Throwable e, Description description), check description.getMethodName() starts with "critical" before logging

B. Override beforeTestExecution(ExtensionContext context) and afterTestExecution(ExtensionContext context), check context.getTestMethod().getName() starts with "critical" before logging

C. Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging

D. Override testStarted(Description description) and testFailure(Failure failure), check failure.getDescription().getMethodName() starts with "critical" before logging

Solution

1. Step 1: Identify correct JUnit 5 TestWatcher methods

   In JUnit 5, TestWatcher methods to override are starting(ExtensionContext) and failed(ExtensionContext, Throwable).

2. Step 2: Filter test names using context.getDisplayName()

   Use context.getDisplayName() to get the test name and check if it starts with "critical" before logging.

3. Final Answer:

   Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging -> Option C

4. Quick Check:

   JUnit 5 TestWatcher + ExtensionContext + name check = Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging [OK]

Hint: Use ExtensionContext methods and check displayName for filtering [OK]

Common Mistakes:

• Using JUnit 4 method signatures in JUnit 5
• Checking wrong parameter types
• Using lifecycle methods not related to TestWatcher