CI pipeline test stage in JUnit - Test Execution Trace

Test Overview

This test runs in a CI pipeline stage to verify that the Calculator's add method works correctly. It checks if adding two numbers returns the expected sum.

Test Code - JUnit 5

JUnit

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

public class CalculatorTest {

    @Test
    public void testAdd() {
        Calculator calc = new Calculator();
        int result = calc.add(2, 3);
        assertEquals(5, result, "2 + 3 should equal 5");
    }
}

class Calculator {
    public int add(int a, int b) {
        return a + b;
    }
}

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Test runner starts and loads CalculatorTest class	JUnit test environment initialized	—	PASS
2	Runs testAdd() method	Calculator instance created	—	PASS
3	Calls add(2, 3) method	Method returns integer result	—	PASS
4	Assert that result equals 5	Result is 5	assertEquals(5, result)	PASS
5	Test completes successfully	Test marked as passed in report	—	PASS

Failure Scenario

Failing Condition: add method returns incorrect sum

Execution Trace Quiz - 3 Questions

What does the testAdd() method verify?

A That adding 2 and 3 returns 5

B That the Calculator class exists

C That the test runner starts correctly

D That the add method throws an exception

Key Result

Always include clear assertions in your CI pipeline tests to catch errors early and provide meaningful feedback.

Practice

1. What is the main purpose of the test stage in a CI pipeline using JUnit?

easy

A. To create documentation for the project

B. To deploy the application to production

C. To merge code branches automatically

D. To run automated tests and verify code correctness

Solution

1. Step 1: Understand the role of the test stage

   The test stage runs automated tests to check if the code works as expected.

2. Step 2: Identify the correct purpose

   Running tests helps catch errors early before code is shared or deployed.

3. Final Answer:

   To run automated tests and verify code correctness -> Option D

4. Quick Check:

   Test stage = run tests [OK]

Hint: Test stage always runs automated tests [OK]

Common Mistakes:

• Confusing test stage with deployment
• Thinking test stage merges code
• Assuming test stage creates docs

2. Which of the following is the correct way to specify a JUnit test command in a CI pipeline script?

easy

A. mvn test

B. npm run test

C. docker build .

D. git commit -m 'test'

Solution

1. Step 1: Identify the build tool for JUnit tests

   JUnit tests are commonly run using Maven with the command mvn test.

2. Step 2: Check other options

   npm run test is for Node.js projects, docker build builds images, and git commit commits code.

3. Final Answer:

   mvn test -> Option A

4. Quick Check:

   JUnit test command = mvn test [OK]

Hint: Use 'mvn test' to run JUnit tests in Maven projects [OK]

Common Mistakes:

• Using npm commands for Java projects
• Confusing build commands with test commands
• Running git commands instead of tests

3. Given this snippet in a CI pipeline YAML:

steps:
  - name: Run tests
    run: mvn test
    continue-on-error: false

What happens if a JUnit test fails during this step?

medium

A. The pipeline stops and marks the test stage as failed

B. The pipeline ignores the failure and continues

C. The pipeline retries the test automatically

D. The pipeline skips the test stage

Solution

1. Step 1: Understand 'continue-on-error: false'

   This setting means the pipeline will stop if the command fails.

2. Step 2: Recognize test failure effect

   If a JUnit test fails, mvn test returns an error, so the pipeline stops and marks failure.

3. Final Answer:

   The pipeline stops and marks the test stage as failed -> Option A

4. Quick Check:

   continue-on-error false + test fail = stop pipeline [OK]

Hint: False continue-on-error stops pipeline on test failure [OK]

Common Mistakes:

• Assuming pipeline continues despite failure
• Thinking tests auto-retry on failure
• Believing test stage is skipped on failure

4. You added a JUnit test stage in your CI pipeline but the test reports are not showing in the pipeline UI. What is the most likely cause?

medium

A. The pipeline skipped the test stage due to a previous failure

B. The test command syntax is incorrect and tests never run

C. JUnit report files are not generated or not saved as artifacts

D. The pipeline configuration file is missing

Solution

1. Step 1: Check test report generation

   JUnit reports must be generated and saved as artifacts for the UI to display them.

2. Step 2: Understand impact of missing reports

   If reports are missing or not saved, the UI cannot show test results even if tests ran.

3. Final Answer:

   JUnit report files are not generated or not saved as artifacts -> Option C

4. Quick Check:

   Missing reports = no UI display [OK]

Hint: Save JUnit reports as artifacts to see results in UI [OK]

Common Mistakes:

• Assuming syntax errors cause missing reports
• Thinking pipeline skipping always hides reports
• Believing missing config file causes this specific issue

5. In a CI pipeline, you want to fail the build if any JUnit test fails but still collect all test reports for review. Which configuration achieves this?

hard

A. Run tests with mvn test, do not save reports, and set continue-on-error: true

B. Run tests with mvn test, save reports as artifacts, and set continue-on-error: false

C. Skip tests and only save previous reports as artifacts

D. Run tests with mvn verify and ignore test failures

Solution

1. Step 1: Ensure tests run and fail build on failure

   Using mvn test with continue-on-error: false stops the build on test failure.

2. Step 2: Save test reports as artifacts

   Saving JUnit reports as artifacts allows review of test results even if build fails.

3. Final Answer:

   Run tests with mvn test, save reports as artifacts, and set continue-on-error: false -> Option B

4. Quick Check:

   Fail build + save reports = mvn test + artifacts + continue-on-error false [OK]

Hint: Fail build but save reports by disabling continue-on-error and saving artifacts [OK]

Common Mistakes:

• Setting continue-on-error true hides failures
• Not saving reports loses test details
• Skipping tests but expecting reports