CI pipeline test stage in JUnit - Build an Automation Script

Verify CI pipeline test stage runs unit tests and reports results

Preconditions (2)

Step 1: Trigger the CI pipeline manually or by pushing code

Step 2: Wait for the test stage to start and complete

Step 3: Check the test stage logs for test execution details

Step 4: Verify that all unit tests are executed

Step 5: Verify that the test results are reported in the CI system

✅ Expected Result: The test stage completes successfully with all unit tests run and results reported without errors

Automation Requirements - JUnit 5

Assertions Needed:

Assert that all expected test methods are executed

Assert that no tests failed

Assert that test results are generated and accessible

Best Practices:

Use @Test annotations for test methods

Use assertions from org.junit.jupiter.api.Assertions

Keep tests independent and repeatable

Use descriptive test method names

Automated Solution

JUnit

import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class SampleUnitTest {

    @Test
    void testAddition() {
        int result = 2 + 3;
        assertEquals(5, result, "2 + 3 should equal 5");
    }

    @Test
    void testStringContains() {
        String text = "Hello CI Pipeline";
        assertTrue(text.contains("CI"), "Text should contain 'CI'");
    }

    @Test
    void testArrayNotEmpty() {
        int[] numbers = {1, 2, 3};
        assertNotEquals(0, numbers.length, "Array should not be empty");
    }

}

This JUnit 5 test class SampleUnitTest contains three simple unit tests.

Each test method is annotated with @Test so the CI pipeline test stage will detect and run them automatically.

Assertions check expected outcomes: assertEquals verifies addition, assertTrue checks string content, and assertNotEquals ensures the array is not empty.

When the CI pipeline runs mvn test, it will execute these tests and report results. If all assertions pass, the test stage will succeed.

This setup ensures the CI pipeline test stage runs unit tests and reports results as expected.

Common Mistakes - 4 Pitfalls

Not using @Test annotation on test methods

Using System.out.println instead of assertions

Writing dependent tests that rely on execution order

Ignoring test failures in CI pipeline logs

Bonus Challenge

Now add data-driven testing with 3 different inputs for a method that checks if a number is even

Show Hint

Practice

1. What is the main purpose of the test stage in a CI pipeline using JUnit?

easy

A. To create documentation for the project

B. To deploy the application to production

C. To merge code branches automatically

D. To run automated tests and verify code correctness

Solution

1. Step 1: Understand the role of the test stage

   The test stage runs automated tests to check if the code works as expected.

2. Step 2: Identify the correct purpose

   Running tests helps catch errors early before code is shared or deployed.

3. Final Answer:

   To run automated tests and verify code correctness -> Option D

4. Quick Check:

   Test stage = run tests [OK]

Hint: Test stage always runs automated tests [OK]

Common Mistakes:

• Confusing test stage with deployment
• Thinking test stage merges code
• Assuming test stage creates docs

2. Which of the following is the correct way to specify a JUnit test command in a CI pipeline script?

easy

A. mvn test

B. npm run test

C. docker build .

D. git commit -m 'test'

Solution

1. Step 1: Identify the build tool for JUnit tests

   JUnit tests are commonly run using Maven with the command mvn test.

2. Step 2: Check other options

   npm run test is for Node.js projects, docker build builds images, and git commit commits code.

3. Final Answer:

   mvn test -> Option A

4. Quick Check:

   JUnit test command = mvn test [OK]

Hint: Use 'mvn test' to run JUnit tests in Maven projects [OK]

Common Mistakes:

• Using npm commands for Java projects
• Confusing build commands with test commands
• Running git commands instead of tests

3. Given this snippet in a CI pipeline YAML:

steps:
  - name: Run tests
    run: mvn test
    continue-on-error: false

What happens if a JUnit test fails during this step?

medium

A. The pipeline stops and marks the test stage as failed

B. The pipeline ignores the failure and continues

C. The pipeline retries the test automatically

D. The pipeline skips the test stage

Solution

1. Step 1: Understand 'continue-on-error: false'

   This setting means the pipeline will stop if the command fails.

2. Step 2: Recognize test failure effect

   If a JUnit test fails, mvn test returns an error, so the pipeline stops and marks failure.

3. Final Answer:

   The pipeline stops and marks the test stage as failed -> Option A

4. Quick Check:

   continue-on-error false + test fail = stop pipeline [OK]

Hint: False continue-on-error stops pipeline on test failure [OK]

Common Mistakes:

• Assuming pipeline continues despite failure
• Thinking tests auto-retry on failure
• Believing test stage is skipped on failure

4. You added a JUnit test stage in your CI pipeline but the test reports are not showing in the pipeline UI. What is the most likely cause?

medium

A. The pipeline skipped the test stage due to a previous failure

B. The test command syntax is incorrect and tests never run

C. JUnit report files are not generated or not saved as artifacts

D. The pipeline configuration file is missing

Solution

1. Step 1: Check test report generation

   JUnit reports must be generated and saved as artifacts for the UI to display them.

2. Step 2: Understand impact of missing reports

   If reports are missing or not saved, the UI cannot show test results even if tests ran.

3. Final Answer:

   JUnit report files are not generated or not saved as artifacts -> Option C

4. Quick Check:

   Missing reports = no UI display [OK]

Hint: Save JUnit reports as artifacts to see results in UI [OK]

Common Mistakes:

• Assuming syntax errors cause missing reports
• Thinking pipeline skipping always hides reports
• Believing missing config file causes this specific issue

5. In a CI pipeline, you want to fail the build if any JUnit test fails but still collect all test reports for review. Which configuration achieves this?

hard

A. Run tests with mvn test, do not save reports, and set continue-on-error: true

B. Run tests with mvn test, save reports as artifacts, and set continue-on-error: false

C. Skip tests and only save previous reports as artifacts

D. Run tests with mvn verify and ignore test failures

Solution

1. Step 1: Ensure tests run and fail build on failure

   Using mvn test with continue-on-error: false stops the build on test failure.

2. Step 2: Save test reports as artifacts

   Saving JUnit reports as artifacts allows review of test results even if build fails.

3. Final Answer:

   Run tests with mvn test, save reports as artifacts, and set continue-on-error: false -> Option B

4. Quick Check:

   Fail build + save reports = mvn test + artifacts + continue-on-error false [OK]

Hint: Fail build but save reports by disabling continue-on-error and saving artifacts [OK]

Common Mistakes:

• Setting continue-on-error true hides failures
• Not saving reports loses test details
• Skipping tests but expecting reports