TestWatcher for reporting in JUnit - Framework Patterns

Framework Mode - TestWatcher for reporting

Folder Structure

src/
└── test/
    └── java/
        └── com/
            └── example/
                ├── tests/
                │   └── LoginTest.java
                ├── watchers/
                │   └── CustomTestWatcher.java
                ├── pages/
                │   └── LoginPage.java
                └── utils/
                    └── TestUtils.java

Test Framework Layers

• Test Classes: Contain test methods using JUnit annotations (@Test). Example: LoginTest.java.
• TestWatcher Layer: Custom classes extending TestWatcher to listen to test events (success, failure, skipped) and perform reporting actions. Example: CustomTestWatcher.java.
• Page Objects: Represent UI pages with methods to interact with elements. Example: LoginPage.java.
• Utilities: Helper classes for common functions like screenshots, logging, or data handling. Example: TestUtils.java.
• Configuration: External files or classes managing environment settings, credentials, and browser options.

Configuration Patterns

• Properties File: Use config.properties to store environment URLs, browser types, and credentials.
• System Properties: Override config values via command line for flexibility (e.g., -Dbrowser=chrome).
• TestWatcher Setup: Initialize reporting tools and screenshot paths in the watcher constructor or setup methods.
• Environment Profiles: Separate config files or sections for dev, test, and prod environments to switch easily.

Test Reporting and CI/CD Integration

• JUnit Reports: Use built-in JUnit XML reports for basic pass/fail results.
• Custom Reporting: Extend TestWatcher to capture screenshots on failure and log detailed messages.
• CI/CD Integration: Configure Jenkins, GitHub Actions, or other pipelines to run tests and publish reports automatically.
• Report Storage: Save screenshots and logs in a structured folder for easy access after test runs.

Best Practices

1. Single Responsibility: Keep TestWatcher focused on reporting and logging, not test logic.
2. Reusable Watchers: Create generic watchers that can be reused across multiple test classes.
3. Clear Naming: Name watcher classes clearly to reflect their purpose, e.g., CustomTestWatcher or ScreenshotOnFailureWatcher.
4. Fail Fast Reporting: Capture failure details immediately to help quick debugging.
5. Integrate with Logs: Combine watcher reports with logging frameworks for comprehensive test insights.

Self Check

Where in this folder structure would you add a new TestWatcher class that captures screenshots on test failure?

Key Result

Use JUnit's TestWatcher to listen to test events and enhance reporting with screenshots and logs.

Practice

1. What is the primary purpose of using TestWatcher in JUnit tests?

easy

A. To react to test lifecycle events like start, success, failure, and finish

B. To write assertions inside test methods

C. To create mock objects for testing

D. To configure database connections before tests

Solution

1. Step 1: Understand TestWatcher role

   TestWatcher is a JUnit class designed to listen to test lifecycle events such as when a test starts, succeeds, fails, or finishes.

2. Step 2: Differentiate from other test tasks

   Writing assertions, mocking, or configuring databases are separate concerns and not the main purpose of TestWatcher.

3. Final Answer:

   To react to test lifecycle events like start, success, failure, and finish -> Option A

4. Quick Check:

   TestWatcher = react to test events [OK]

Hint: TestWatcher listens to test events, not assertions or mocks [OK]

Common Mistakes:

• Confusing TestWatcher with assertion writing
• Thinking TestWatcher creates mocks
• Assuming TestWatcher manages test setup like DB connections

2. Which method signature correctly overrides the TestWatcher method that runs when a test fails?

easy

A. public void failed(ExtensionContext context, Throwable cause)

B. protected void failed(Throwable e, Description description)

C. void failed(Description description, Throwable e)

D. public void testFailed(ExtensionContext context, Throwable cause)

Solution

1. Step 1: Recall TestWatcher method signature

   The failed method in TestWatcher is public void failed(ExtensionContext context, Throwable cause).

2. Step 2: Check method modifiers and parameter order

   It must be public, return void, and parameters are ExtensionContext first, then Throwable.

3. Final Answer:

   public void failed(ExtensionContext context, Throwable cause) -> Option A

4. Quick Check:

   failed method = public void failed(ExtensionContext, Throwable) [OK]

Hint: Check parameter order and access modifier for overrides [OK]

Common Mistakes:

• Swapping parameter order
• Using protected instead of public
• Missing return type or wrong method name

3. Given the following TestWatcher subclass, what will be printed if the test exampleTest fails?

class MyWatcher extends TestWatcher {
  @Override
  public void failed(ExtensionContext context, Throwable cause) {
    context.getTestMethod().ifPresent(method -> System.out.println(method.getName() + " failed"));
  }
}

@Test
@ExtendWith(MyWatcher.class)
void exampleTest() {
  assertEquals(1, 2);
}

medium

A. No output printed

B. failed

C. exampleTest

D. exampleTest failed

Solution

1. Step 1: Understand what failed() prints

   The overridden failed method prints the test method name plus " failed".

2. Step 2: Identify test method name on failure

   The test method is named exampleTest, so the output will be "exampleTest failed".

3. Final Answer:

   exampleTest failed -> Option D

4. Quick Check:

   Print = methodName + " failed" = exampleTest failed [OK]

Hint: Print uses description.getMethodName() on failure [OK]

Common Mistakes:

• Assuming no output on failure
• Printing only 'failed' without method name
• Confusing method name with test class name

4. Identify the error in this TestWatcher subclass that aims to log test success:

class SuccessWatcher extends TestWatcher {
  @Override
  public void succeeded(Description description) {
    System.out.println(description.getDisplayName() + " succeeded");
  }
}

medium

A. Method signature is incorrect; it should include Throwable parameter

B. Method signature is incorrect; it should be public void succeeded(ExtensionContext context)

C. Method should be protected, not public

D. Method signature is incorrect; it should be public void succeeded(Description description, Throwable e)

Solution

1. Step 1: Check JUnit version and method signature

   In JUnit 5, TestWatcher methods use ExtensionContext parameter, not Description.

2. Step 2: Identify correct succeeded() signature

   The correct override is public void succeeded(ExtensionContext context) in JUnit 5.

3. Final Answer:

   Method signature is incorrect; it should be public void succeeded(ExtensionContext context) -> Option B

4. Quick Check:

   JUnit 5 TestWatcher uses ExtensionContext parameter [OK]

Hint: JUnit 5 TestWatcher methods use ExtensionContext, not Description [OK]

Common Mistakes:

• Using JUnit 4 signatures in JUnit 5
• Wrong parameter types or order
• Confusing access modifiers

5. You want to create a custom TestWatcher that logs test start and failure events, but only for tests whose names start with "critical". Which approach correctly implements this behavior in JUnit 5?

hard

A. Override starting(Description description) and failed(Throwable e, Description description), check description.getMethodName() starts with "critical" before logging

B. Override beforeTestExecution(ExtensionContext context) and afterTestExecution(ExtensionContext context), check context.getTestMethod().getName() starts with "critical" before logging

C. Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging

D. Override testStarted(Description description) and testFailure(Failure failure), check failure.getDescription().getMethodName() starts with "critical" before logging

Solution

1. Step 1: Identify correct JUnit 5 TestWatcher methods

   In JUnit 5, TestWatcher methods to override are starting(ExtensionContext) and failed(ExtensionContext, Throwable).

2. Step 2: Filter test names using context.getDisplayName()

   Use context.getDisplayName() to get the test name and check if it starts with "critical" before logging.

3. Final Answer:

   Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging -> Option C

4. Quick Check:

   JUnit 5 TestWatcher + ExtensionContext + name check = Override starting(ExtensionContext context) and failed(ExtensionContext context, Throwable cause), check context.getDisplayName() starts with "critical" before logging [OK]

Hint: Use ExtensionContext methods and check displayName for filtering [OK]

Common Mistakes:

• Using JUnit 4 method signatures in JUnit 5
• Checking wrong parameter types
• Using lifecycle methods not related to TestWatcher