Code coverage reports in Jenkins - Commands & Configuration

Introduction

Code coverage reports show how much of your code is tested by automated tests. They help find untested parts so you can improve your tests and catch bugs early.

When you want to check if your automated tests cover most of your code.

When you want to prevent untested code from being added to your project.

When you want to track test coverage over time to improve software quality.

When you want to fail a build if coverage drops below a certain level.

When you want to generate visual reports to share with your team.

Config File - Jenkinsfile

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Build') {
      steps {
        echo 'Building the project...'
        // Add build commands here
      }
    }
    stage('Test') {
      steps {
        echo 'Running tests...'
        // Run tests with coverage enabled, e.g., using Maven or Gradle
        sh 'mvn clean test'
      }
      post {
        always {
          junit '**/target/surefire-reports/*.xml'
          cobertura coberturaReportFile: '**/target/site/cobertura/coverage.xml'
        }
      }
    }
  }
}

This Jenkinsfile defines a pipeline with two stages: Build and Test.

In the Test stage, it runs tests with coverage enabled (example uses Maven).

After tests, it publishes test results with junit and coverage reports with cobertura plugin.

The coberturaReportFile points to the coverage XML file generated by the test tool.

Commands

Runs the tests and generates the code coverage report using Maven with the coverage plugin enabled.

Terminal

mvn clean test

Expected Output

[INFO] Scanning for projects...

[INFO] 

[INFO] ------------------< com.example:my-app >-------------------

[INFO] Building my-app 1.0-SNAPSHOT

[INFO] --------------------------------[ jar ]---------------------------------

[INFO] 

[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ my-app ---

[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.345 s - in com.example.AppTest

[INFO] 

[INFO] --- cobertura-maven-plugin:2.7:check (default) @ my-app ---

[INFO] Cobertura coverage: 85%

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time:  10.123 s

[INFO] Finished at: 2024-06-01T12:00:00Z

[INFO] ------------------------------------------------------------------------

Triggers the Jenkins job named 'my-app' to start the pipeline that includes running tests and generating coverage reports.

Terminal

jenkins-jobs build my-app

Expected Output

Started build #42 for job my-app

Building in workspace /var/lib/jenkins/workspace/my-app

[Pipeline] stage

[Pipeline] { (Build)

[Pipeline] echo

Building the project...

[Pipeline] }

[Pipeline] // stage

[Pipeline] stage

[Pipeline] { (Test)

[Pipeline] echo

Running tests...

[Pipeline] sh

+ mvn clean test

[INFO] Scanning for projects...

...

[INFO] BUILD SUCCESS

[Pipeline] junit

Recording test results

[Pipeline] cobertura

Publishing Cobertura coverage report

[Pipeline] }

[Pipeline] // stage

[Pipeline] End of Pipeline

Finished: SUCCESS

Fetches the HTML report page for the code coverage of build number 42 from Jenkins to view coverage details in the browser.

Terminal

curl http://jenkins.example.com/job/my-app/42/cobertura/

Expected Output

<html>

<head><title>Code Coverage Report</title></head>

<body>

<h1>Coverage Report for Build #42</h1>

<p>Line Coverage: 85%</p>

<p>Branch Coverage: 80%</p>

<!-- More detailed coverage info here -->

</body>

</html>

Key Concept

If you remember nothing else from this pattern, remember: code coverage reports show which parts of your code are tested, helping you improve test quality and software reliability.

Common Mistakes

Not generating the coverage report file before publishing it in Jenkins.

Jenkins cannot find the coverage XML file, so the report step fails or shows no data.

Ensure your test tool runs with coverage enabled and generates the coverage XML file before Jenkins tries to publish it.

Using incorrect file paths for the coverage report in the Jenkinsfile.

Jenkins cannot locate the coverage report file, so the coverage plugin shows errors or empty reports.

Use the exact relative path to the coverage XML file generated by your test tool in the Jenkins workspace.

Not installing or enabling the Jenkins coverage plugin (e.g., Cobertura plugin).

Jenkins does not recognize the coverage report step and fails the pipeline.

Install and enable the appropriate coverage plugin in Jenkins before using coverage report steps.

Summary

Run tests with coverage enabled to generate coverage report files.

Use Jenkins pipeline steps to publish test results and coverage reports.

View coverage reports in Jenkins UI or fetch them via HTTP for review.

Practice

1. What is the main purpose of code coverage reports in Jenkins?

easy

A. To show which parts of the code are tested

B. To deploy the application automatically

C. To monitor server CPU usage

D. To manage user permissions

Solution

1. Step 1: Understand code coverage reports

   Code coverage reports indicate which parts of the source code have been executed during tests.

2. Step 2: Identify Jenkins role

   Jenkins uses these reports to help developers see test completeness and improve code quality.

3. Final Answer:

   To show which parts of the code are tested -> Option A

4. Quick Check:

   Code coverage = tested code parts [OK]

Hint: Coverage reports show tested code areas, not deployment [OK]

Common Mistakes:

• Confusing coverage with deployment
• Thinking coverage monitors server stats
• Mixing coverage with user management

2. Which Jenkins plugin is commonly used to publish Java code coverage reports?

easy

A. Git Plugin

B. JaCoCo Plugin

C. Docker Plugin

D. Slack Notification Plugin

Solution

1. Step 1: Identify coverage plugins for Java

   JaCoCo is a popular plugin for Java projects to generate code coverage reports in Jenkins.

2. Step 2: Exclude unrelated plugins

   Git manages source code, Docker manages containers, Slack sends messages, so they don't handle coverage.

3. Final Answer:

   JaCoCo Plugin -> Option B

4. Quick Check:

   Java coverage = JaCoCo [OK]

Hint: JaCoCo is the standard for Java coverage in Jenkins [OK]

Common Mistakes:

• Choosing Git for coverage
• Confusing Docker with coverage tools
• Selecting notification plugins

3. Given this Jenkins pipeline snippet, what will be the result of the jacoco step?

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        junit '**/test-results.xml'
        jacoco execPattern: '**/jacoco.exec'
      }
    }
  }
}

medium

A. Publishes JaCoCo coverage report using the specified exec file

B. Runs unit tests and skips coverage report

C. Fails because jacoco is not a valid step

D. Publishes test results but ignores coverage data

Solution

1. Step 1: Understand the jacoco step usage

   The jacoco step in Jenkins pipeline publishes coverage reports using the exec file pattern provided.

2. Step 2: Analyze the snippet

   The pipeline runs tests with junit and then publishes coverage with jacoco execPattern: '**/jacoco.exec'.

3. Final Answer:

   Publishes JaCoCo coverage report using the specified exec file -> Option A

4. Quick Check:

   jacoco step publishes coverage report [OK]

Hint: jacoco step publishes coverage from exec files [OK]

Common Mistakes:

• Thinking jacoco runs tests
• Assuming jacoco step is invalid
• Ignoring coverage data in pipeline

4. You added the JaCoCo plugin to Jenkins but the coverage report does not appear after the build. What is the most likely cause?

medium

A. The Jenkins server is offline

B. The Jenkins user lacks admin rights

C. The build script does not run any tests

D. The jacoco.exec file path is incorrect or missing

Solution

1. Step 1: Check coverage report prerequisites

   JaCoCo needs the jacoco.exec file generated by tests to publish coverage.

2. Step 2: Identify common error

   If the exec file path is wrong or the file is missing, Jenkins cannot show the coverage report.

3. Final Answer:

   The jacoco.exec file path is incorrect or missing -> Option D

4. Quick Check:

   Missing exec file = no coverage report [OK]

Hint: Check jacoco.exec file path if coverage missing [OK]

Common Mistakes:

• Blaming Jenkins offline status
• Ignoring test execution for coverage
• Assuming user rights block coverage display

5. You want Jenkins to fail the build if code coverage falls below 80%. Which configuration should you add to the JaCoCo plugin in your pipeline?

hard

A. jacoco(execPattern: '**/jacoco.exec', failBuildIfCoverageLow: 80)

B. jacoco(execPattern: '**/jacoco.exec', changeBuildStatus: true, minimumCoverage: 80)

C. jacoco(execPattern: '**/jacoco.exec', changeBuildStatus: true, minimumInstructionCoverage: 80)

D. jacoco(execPattern: '**/jacoco.exec', coverageThreshold: 80)

Solution

1. Step 1: Understand JaCoCo coverage thresholds

   The JaCoCo plugin uses minimumInstructionCoverage to set the minimum coverage percentage for instructions.

2. Step 2: Identify correct syntax to fail build

   Setting changeBuildStatus: true with minimumInstructionCoverage: 80 makes Jenkins fail if coverage is below 80%.

3. Final Answer:

   jacoco(execPattern: '**/jacoco.exec', changeBuildStatus: true, minimumInstructionCoverage: 80) -> Option C

4. Quick Check:

   minimumInstructionCoverage + changeBuildStatus = fail below threshold [OK]

Hint: Use minimumInstructionCoverage with changeBuildStatus true [OK]

Common Mistakes:

• Using non-existent parameters like coverageThreshold
• Omitting changeBuildStatus to fail build
• Confusing minimumCoverage with minimumInstructionCoverage