CI pipeline test stage in JUnit - Framework Patterns

Framework Mode - CI pipeline test stage

Folder Structure

my-project/
├── src/
│   ├── main/
│   │   └── java/
│   │       └── com/example/app/
│   │           └── Application.java
│   └── test/
│       └── java/
│           └── com/example/app/
│               ├── pages/
│               │   └── LoginPage.java
│               ├── tests/
│               │   └── LoginTest.java
│               └── utils/
│                   └── TestUtils.java
├── pom.xml
├── Jenkinsfile
└── README.md
    

Test Framework Layers

• Driver Layer: Manages WebDriver setup and teardown for browser automation.
• Page Objects: Classes representing UI pages with methods to interact with page elements.
• Tests: JUnit test classes containing test methods with assertions.
• Utilities: Helper classes for common functions like waits, data handling, and logging.
• Configuration: External files and code to manage environment variables, browser types, and credentials.

Configuration Patterns

Use pom.xml for dependencies and plugins.

Use src/test/resources/config.properties to store environment URLs, browser names, and credentials securely.

Use system properties or environment variables to switch environments and browsers during CI runs.

Example snippet in pom.xml to pass parameters:

<properties>
  <env>staging</env>
  <browser>chrome</browser>
</properties>

In tests, read these properties to configure test execution dynamically.

Test Reporting and CI/CD Integration

• Use JUnit's built-in XML reports for test results.
• Integrate with Jenkins or other CI tools to run tests automatically on code commits.
• Configure Jenkinsfile to include a test stage that runs mvn test and archives reports.
• Use plugins like JUnit Plugin in Jenkins to visualize test results and trends.
• Fail the pipeline if tests fail to prevent bad code from deploying.

Best Practices

1. Keep tests isolated and independent to avoid flaky failures.
2. Use explicit waits to handle dynamic web elements reliably.
3. Store sensitive data like credentials outside source code, use environment variables or secure vaults.
4. Run tests in parallel in CI to reduce feedback time.
5. Keep the CI pipeline fast by running only necessary tests on each commit.

Self Check

Question: In this framework structure, where would you add a new page object class for the "Dashboard" page?

Answer: Add the new DashboardPage.java class inside the src/test/java/com/example/app/pages/ folder.

Key Result

Organize tests with clear layers, configure environments flexibly, and integrate automated test runs in CI pipelines for reliable feedback.

Practice

1. What is the main purpose of the test stage in a CI pipeline using JUnit?

easy

A. To create documentation for the project

B. To deploy the application to production

C. To merge code branches automatically

D. To run automated tests and verify code correctness

Solution

1. Step 1: Understand the role of the test stage

   The test stage runs automated tests to check if the code works as expected.

2. Step 2: Identify the correct purpose

   Running tests helps catch errors early before code is shared or deployed.

3. Final Answer:

   To run automated tests and verify code correctness -> Option D

4. Quick Check:

   Test stage = run tests [OK]

Hint: Test stage always runs automated tests [OK]

Common Mistakes:

• Confusing test stage with deployment
• Thinking test stage merges code
• Assuming test stage creates docs

2. Which of the following is the correct way to specify a JUnit test command in a CI pipeline script?

easy

A. mvn test

B. npm run test

C. docker build .

D. git commit -m 'test'

Solution

1. Step 1: Identify the build tool for JUnit tests

   JUnit tests are commonly run using Maven with the command mvn test.

2. Step 2: Check other options

   npm run test is for Node.js projects, docker build builds images, and git commit commits code.

3. Final Answer:

   mvn test -> Option A

4. Quick Check:

   JUnit test command = mvn test [OK]

Hint: Use 'mvn test' to run JUnit tests in Maven projects [OK]

Common Mistakes:

• Using npm commands for Java projects
• Confusing build commands with test commands
• Running git commands instead of tests

3. Given this snippet in a CI pipeline YAML:

steps:
  - name: Run tests
    run: mvn test
    continue-on-error: false

What happens if a JUnit test fails during this step?

medium

A. The pipeline stops and marks the test stage as failed

B. The pipeline ignores the failure and continues

C. The pipeline retries the test automatically

D. The pipeline skips the test stage

Solution

1. Step 1: Understand 'continue-on-error: false'

   This setting means the pipeline will stop if the command fails.

2. Step 2: Recognize test failure effect

   If a JUnit test fails, mvn test returns an error, so the pipeline stops and marks failure.

3. Final Answer:

   The pipeline stops and marks the test stage as failed -> Option A

4. Quick Check:

   continue-on-error false + test fail = stop pipeline [OK]

Hint: False continue-on-error stops pipeline on test failure [OK]

Common Mistakes:

• Assuming pipeline continues despite failure
• Thinking tests auto-retry on failure
• Believing test stage is skipped on failure

4. You added a JUnit test stage in your CI pipeline but the test reports are not showing in the pipeline UI. What is the most likely cause?

medium

A. The pipeline skipped the test stage due to a previous failure

B. The test command syntax is incorrect and tests never run

C. JUnit report files are not generated or not saved as artifacts

D. The pipeline configuration file is missing

Solution

1. Step 1: Check test report generation

   JUnit reports must be generated and saved as artifacts for the UI to display them.

2. Step 2: Understand impact of missing reports

   If reports are missing or not saved, the UI cannot show test results even if tests ran.

3. Final Answer:

   JUnit report files are not generated or not saved as artifacts -> Option C

4. Quick Check:

   Missing reports = no UI display [OK]

Hint: Save JUnit reports as artifacts to see results in UI [OK]

Common Mistakes:

• Assuming syntax errors cause missing reports
• Thinking pipeline skipping always hides reports
• Believing missing config file causes this specific issue

5. In a CI pipeline, you want to fail the build if any JUnit test fails but still collect all test reports for review. Which configuration achieves this?

hard

A. Run tests with mvn test, do not save reports, and set continue-on-error: true

B. Run tests with mvn test, save reports as artifacts, and set continue-on-error: false

C. Skip tests and only save previous reports as artifacts

D. Run tests with mvn verify and ignore test failures

Solution

1. Step 1: Ensure tests run and fail build on failure

   Using mvn test with continue-on-error: false stops the build on test failure.

2. Step 2: Save test reports as artifacts

   Saving JUnit reports as artifacts allows review of test results even if build fails.

3. Final Answer:

   Run tests with mvn test, save reports as artifacts, and set continue-on-error: false -> Option B

4. Quick Check:

   Fail build + save reports = mvn test + artifacts + continue-on-error false [OK]

Hint: Fail build but save reports by disabling continue-on-error and saving artifacts [OK]

Common Mistakes:

• Setting continue-on-error true hides failures
• Not saving reports loses test details
• Skipping tests but expecting reports