Test result trends in Jenkins - Commands & Configuration

Introduction

When you run tests in Jenkins, you want to see how your test results change over time. Test result trends help you spot if tests are getting better or worse. This helps keep your software healthy and reliable.

When you want to track if new code breaks existing tests over several builds

When you want to see if test failures are increasing or decreasing over time

When you want to quickly find builds with the most test failures

When you want to share test quality reports with your team automatically

When you want to improve your testing process by analyzing test trends

Commands

This command creates or updates a Jenkins job from a YAML file that includes test reporting configuration. It sets up the job to run tests and collect results.

Terminal

jenkins-jobs --conf jenkins.ini test-job.yaml

Expected Output

Job 'test-job' created or updated successfully

→

--conf - Specifies the Jenkins server configuration file

This command triggers the Jenkins job named 'test-job' and waits for it to finish. It runs the tests and collects the results.

Terminal

jenkins-cli build test-job -s

Expected Output

Started build #1

Finished: SUCCESS

→

-s - Waits for the build to complete before returning

This command shows the console output of build number 1 of the 'test-job'. You can check test execution details here.

Terminal

jenkins-cli console test-job 1

Expected Output

[Pipeline] Start of Pipeline

Running tests...

Tests passed: 10, failed: 0

[Pipeline] End of Pipeline

Finished: SUCCESS

This command retrieves the job configuration XML. You can verify that the test reporting plugin is configured to collect test results for trend analysis.

Terminal

jenkins-cli get-job test-job

Expected Output

<?xml version="1.0" encoding="UTF-8"?>

<project>

  <publishers>

    <hudson.tasks.junit.JUnitResultArchiver>

      <testResults>**/test-results.xml</testResults>

      <keepLongStdio>false</keepLongStdio>

      <healthScaleFactor>1.0</healthScaleFactor>

    </hudson.tasks.junit.JUnitResultArchiver>

  </publishers>

</project>

Key Concept

If you remember nothing else from this pattern, remember: Jenkins collects test results each build and shows trends over time to help you track test health.

Common Mistakes

Not configuring the test result archiver plugin in the Jenkins job

Without this, Jenkins won't collect test results and cannot show trends.

Add the JUnit test result archiver step in the job configuration pointing to the test result files.

Running tests but not publishing the test result files

Jenkins needs the test result files to generate trend reports; missing files mean no data.

Ensure your test runner outputs test results in a format Jenkins understands and that the job archives these files.

Triggering builds without waiting for completion before checking results

You might check results before tests finish, leading to incomplete or missing data.

Use flags or commands that wait for the build to finish before accessing test results.

Summary

Create or update a Jenkins job configured to run tests and archive test results.

Trigger the Jenkins job and wait for it to complete to ensure test results are collected.

View the console output and job configuration to verify tests ran and results are archived.

Jenkins uses archived test results to show test result trends over multiple builds.