Test result trends in Jenkins - Step-by-Step Execution

Process Flow - Test result trends

Run Tests

↓

Collect Test Results

↓

Store Results in Jenkins

↓

Analyze Trends Over Time

↓

Display Graphs & Reports

↓

Use Trends to Improve Code Quality

This flow shows how Jenkins runs tests, collects results, stores them, analyzes trends over time, and displays reports to help improve code quality.

Execution Sample

Jenkins

pipeline {
  stages {
    stage('Test') {
      steps {
        junit 'reports/*.xml'
      }
    }
  }
}

This Jenkins pipeline runs tests and collects test results from XML files to track test trends.

Process Table

Step	Action	Input/Condition	Result	Notes
1	Run Tests	Codebase ready	Tests executed	Tests run on current code
2	Collect Test Results	Test output files exist	Results parsed	JUnit plugin reads XML reports
3	Store Results	Results parsed	Results saved in Jenkins	Data stored for trend analysis
4	Analyze Trends	Multiple builds with results	Trend graphs generated	Shows pass/fail over time
5	Display Reports	Trend graphs ready	Reports visible in Jenkins UI	Users see test stability
6	Use Trends	Reports viewed	Decisions made to improve tests	Helps fix flaky or failing tests
7	End	No more builds	Monitoring continues	Process repeats with each build

💡 Process repeats after each build to update test result trends

Status Tracker

Variable	Start	After Step 1	After Step 2	After Step 3	After Step 4	After Step 5	Final
Tests Run	0	Executed	Executed	Executed	Executed	Executed	Executed
Results Parsed	No	No	Yes	Yes	Yes	Yes	Yes
Results Stored	No	No	No	Yes	Yes	Yes	Yes
Trend Graphs	None	None	None	Generated	Displayed	Displayed	Displayed
Reports Visible	No	No	No	No	Yes	Yes	Yes

Key Moments - 3 Insights

Why do test results need to be stored in Jenkins after parsing?

Storing results (see step 3 in execution_table) allows Jenkins to keep historical data needed to analyze trends over multiple builds.

What triggers the generation of trend graphs?

Trend graphs are generated only after multiple builds have test results stored (step 4), so Jenkins can compare results over time.

How does viewing reports help improve code quality?

Viewing reports (step 6) helps identify flaky or failing tests, so developers can fix them and improve overall test stability.

Visual Quiz - 3 Questions

Look at the execution_table, at which step are test results parsed from XML files?

A Step 4

B Step 3

C Step 2

D Step 5

💡 Hint

Check the 'Action' and 'Result' columns in execution_table row for step 2.

According to variable_tracker, when do trend graphs first appear?

A After Step 3

B After Step 4

C After Step 5

D After Step 6

💡 Hint

Look at the 'Trend Graphs' row and see when it changes from 'None' to 'Generated'.

If no test results are stored, what will happen to trend analysis?

A Trend analysis cannot proceed

B Trend graphs will still be generated

C Reports will be displayed without data

D Tests will rerun automatically

💡 Hint

Refer to step 3 and 4 in execution_table about storing results and generating trends.

Concept Snapshot

Jenkins Test Result Trends:
- Run tests and generate XML reports
- Use 'junit' step to collect results
- Jenkins stores results per build
- Trend graphs show pass/fail over time
- Use trends to find flaky tests and improve quality

Full Transcript

In Jenkins, test result trends start by running tests and generating XML reports. The 'junit' step collects these results and Jenkins stores them for each build. Over multiple builds, Jenkins analyzes the stored results to generate trend graphs showing test pass and fail rates over time. These reports help developers identify unstable tests and improve code quality. The process repeats with every build to keep trends updated.

Practice

1. What is the main purpose of test result trends in Jenkins?

easy

A. To show how test results change over time

B. To run tests faster

C. To deploy applications automatically

D. To create new test cases

Solution

1. Step 1: Understand test result trends concept

   Test result trends track test outcomes across multiple builds to see progress or issues.

2. Step 2: Identify the main purpose

   The main goal is to observe changes in test results over time, helping catch problems early.

3. Final Answer:

   To show how test results change over time -> Option A

4. Quick Check:

   Test result trends = show changes over time [OK]

Hint: Trends track test results over builds, not speed or deployment [OK]

Common Mistakes:

• Confusing trends with deployment automation
• Thinking trends create tests
• Assuming trends speed up tests

2. Which Jenkins plugin is required to publish test result trends from JUnit XML reports?

easy

A. Git plugin

B. Docker plugin

C. Pipeline plugin

D. JUnit plugin

Solution

1. Step 1: Identify plugin for JUnit XML reports

   JUnit plugin in Jenkins processes JUnit XML test reports and generates test result trends.

2. Step 2: Confirm plugin role

   Only the JUnit plugin handles test result publishing and trend visualization from JUnit reports.

3. Final Answer:

   JUnit plugin -> Option D

4. Quick Check:

   JUnit plugin publishes test trends [OK]

Hint: JUnit plugin handles JUnit XML test reports and trends [OK]

Common Mistakes:

• Choosing Git plugin which manages source code
• Selecting Docker plugin unrelated to tests
• Confusing Pipeline plugin with test reporting

3. Given this Jenkins pipeline snippet, what will happen if the junit '**/test-results.xml' step is missing?

pipeline {
  agent any
  stages {
    stage('Test') {
      steps {
        sh 'run-tests.sh'
        junit '**/test-results.xml'
      }
    }
  }
}

medium

A. Test results will be published and trends shown

B. No test results will be published, no trends available

C. Pipeline will fail with syntax error

D. Tests will not run at all

Solution

1. Step 1: Understand the role of junit step

   The junit step publishes test results from XML files and enables trend graphs.

2. Step 2: Analyze missing junit step effect

   Without junit, test results are not published, so no trends appear, but tests still run.

3. Final Answer:

   No test results will be published, no trends available -> Option B

4. Quick Check:

   Missing junit step = no published results [OK]

Hint: Without junit step, no test results or trends appear [OK]

Common Mistakes:

• Thinking pipeline fails without junit step
• Assuming tests won't run without junit
• Believing trends show without publishing results

4. You added the junit '**/results.xml' step but Jenkins shows no test trends. What is the most likely cause?

medium

A. The XML file path pattern is incorrect or no files match

B. The Jenkins server is offline

C. The pipeline syntax is invalid

D. The tests did not run

Solution

1. Step 1: Check file path pattern for test reports

   If the pattern **/results.xml does not match any files, no results are published.

2. Step 2: Confirm Jenkins behavior on missing files

   Jenkins silently skips publishing if no matching files found, so no trends appear.

3. Final Answer:

   The XML file path pattern is incorrect or no files match -> Option A

4. Quick Check:

   Wrong file path = no test trends [OK]

Hint: Verify XML file path matches actual test report location [OK]

Common Mistakes:

• Assuming Jenkins is offline without checking
• Blaming pipeline syntax without error
• Thinking tests must fail to hide trends

5. You want to track test result trends for multiple test suites in Jenkins, each producing separate JUnit XML files in different folders. How should you configure the junit step to show combined trends?

hard

A. Manually merge XML files before publishing

B. Use multiple junit steps, one per folder

C. Use a single junit step with a combined glob pattern like '**/test-results/*.xml'

D. Publish only one folder's results to avoid confusion

Solution

1. Step 1: Understand how junit step handles multiple files

   The junit step accepts glob patterns to include multiple XML files from different folders.

2. Step 2: Choose best practice for combined trends

   Using a single junit step with a combined pattern like '**/test-results/*.xml' publishes all results together, enabling combined trends.

3. Final Answer:

   Use a single junit step with a combined glob pattern like '**/test-results/*.xml' -> Option C

4. Quick Check:

   Single junit step + combined pattern = combined trends [OK]

Hint: Use one junit step with glob pattern to include all XML files [OK]

Common Mistakes:

• Using multiple junit steps causing separate trends
• Trying to merge XML files manually
• Ignoring some test suites results