Performance test reporting in Testing Fundamentals - Test Execution Trace

Test Overview

This test simulates running a performance test on a web application and verifies that the performance report is generated correctly with key metrics like response time and throughput.

Test Code - unittest

Testing Fundamentals

import unittest

class PerformanceReportTest(unittest.TestCase):
    def test_performance_report_generation(self):
        # Simulate performance test results
        performance_data = {
            'average_response_time_ms': 120,
            'max_response_time_ms': 300,
            'throughput_rps': 50,
            'error_rate_percent': 0.5
        }

        # Generate report string
        report = (f"Average Response Time: {performance_data['average_response_time_ms']} ms\n"
                  f"Max Response Time: {performance_data['max_response_time_ms']} ms\n"
                  f"Throughput: {performance_data['throughput_rps']} requests/sec\n"
                  f"Error Rate: {performance_data['error_rate_percent']}%")

        # Verify report contains all key metrics
        self.assertIn('Average Response Time', report)
        self.assertIn('Max Response Time', report)
        self.assertIn('Throughput', report)
        self.assertIn('Error Rate', report)

        # Verify values are correct
        self.assertIn('120 ms', report)
        self.assertIn('300 ms', report)
        self.assertIn('50 requests/sec', report)
        self.assertIn('0.5%', report)

if __name__ == '__main__':
    unittest.main()

Execution Trace - 12 Steps

Step	Action	System State	Assertion	Result
1	Test starts	Test framework initializes the test case	—	PASS
2	Simulate performance test results with average response time 120 ms, max 300 ms, throughput 50 rps, error rate 0.5%	Performance data dictionary created in memory	—	PASS
3	Generate performance report string with all metrics	Report string contains all performance metrics formatted	—	PASS
4	Check report contains 'Average Response Time'	Report string is available	assertIn('Average Response Time', report)	PASS
5	Check report contains 'Max Response Time'	Report string is available	assertIn('Max Response Time', report)	PASS
6	Check report contains 'Throughput'	Report string is available	assertIn('Throughput', report)	PASS
7	Check report contains 'Error Rate'	Report string is available	assertIn('Error Rate', report)	PASS
8	Verify '120 ms' value is in report	Report string is available	assertIn('120 ms', report)	PASS
9	Verify '300 ms' value is in report	Report string is available	assertIn('300 ms', report)	PASS
10	Verify '50 requests/sec' value is in report	Report string is available	assertIn('50 requests/sec', report)	PASS
11	Verify '0.5%' value is in report	Report string is available	assertIn('0.5%', report)	PASS
12	Test ends successfully	All assertions passed, test completes	—	PASS

Failure Scenario

Failing Condition: Performance report string is missing one or more key metrics or values

Execution Trace Quiz - 3 Questions

What does the test verify about the performance report?

A It measures the actual response time of the application

B It checks the UI layout of the report page

C It contains all key performance metrics with correct values

D It verifies database connection speed

Key Result

Always verify that performance test reports include all important metrics with correct values to ensure accurate and useful results.

Practice

1. What is the main purpose of a performance test report?

easy

A. To list all the bugs found during testing

B. To explain the software installation steps

C. To describe the user interface design

D. To show how software behaves under different loads

Solution

1. Step 1: Understand the role of performance testing

   Performance testing checks how software works when many users or tasks run at once.

2. Step 2: Identify what the report shows

   The report summarizes results like speed, response time, and stability under load.

3. Final Answer:

   To show how software behaves under different loads -> Option D

4. Quick Check:

   Performance test report = software behavior under load [OK]

Hint: Performance reports focus on load behavior, not bugs or UI [OK]

Common Mistakes:

• Confusing performance reports with bug reports
• Thinking reports describe UI or installation
• Ignoring load and speed details

2. Which element is NOT typically included in a performance test report?

easy

A. Steps to reproduce a UI bug

B. Number of concurrent users tested

C. Graphs showing response times

D. Summary of test conclusions

Solution

1. Step 1: Review common report contents

   Performance reports usually have graphs, user counts, and conclusions about performance.

2. Step 2: Identify unrelated content

   Steps to reproduce UI bugs belong in functional or bug reports, not performance reports.

3. Final Answer:

   Steps to reproduce a UI bug -> Option A

4. Quick Check:

   UI bug steps ≠ performance report content [OK]

Hint: Performance reports focus on load data, not UI bugs [OK]

Common Mistakes:

• Including bug reproduction steps in performance reports
• Confusing test conclusion with bug details
• Ignoring graphs and user counts

3. Given this snippet from a performance test report:

Average response time: 2.5 seconds
Peak users: 1000
Error rate: 0.5%

What does the error rate tell us?

easy

A. The percentage of requests that failed during the test

B. The average time users waited for a response

C. The total number of users tested

D. The maximum response time recorded

Solution

1. Step 1: Understand error rate meaning

   Error rate shows how many requests or actions failed compared to total attempts.

2. Step 2: Match error rate to options

   It is a percentage of failures, not time or user count.

3. Final Answer:

   The percentage of requests that failed during the test -> Option A

4. Quick Check:

   Error rate = % of failed requests [OK]

Hint: Error rate means failed requests percentage [OK]

Common Mistakes:

• Confusing error rate with response time
• Mixing error rate with user count
• Thinking error rate is max response time

4. A performance test report shows response times increasing steadily, but the test script mistakenly used only 10 users instead of 1000. What is the main issue?

medium

A. Graphs should not show response times

B. Response times are always expected to increase

C. Test load is too low to reflect real performance

D. Error rate must be zero for valid results

Solution

1. Step 1: Analyze test load impact

   Using only 10 users instead of 1000 means the load is too small to simulate real conditions.

2. Step 2: Understand effect on results

   Low load can hide performance problems or give misleading response times.

3. Final Answer:

   Test load is too low to reflect real performance -> Option C

4. Quick Check:

   Low user load = unreliable performance data [OK]

Hint: Check user count matches test plan for valid results [OK]

Common Mistakes:

• Assuming response times always increase
• Ignoring incorrect user load
• Expecting zero error rate always

5. You have a performance test report with these results:

Average response time: 1.2s
Peak response time: 5.0s
Error rate: 3%
Concurrent users: 500

Which action should you recommend to improve the system?

hard

A. Reduce the number of concurrent users to 100 to lower errors

B. Investigate causes of high peak response times and errors under load

C. Ignore errors since average response time is low

D. Increase test duration without changing load

Solution

1. Step 1: Analyze report data

   Peak response time is much higher than average, and error rate is 3%, which is significant under 500 users.

2. Step 2: Decide best improvement action

   Investigating causes of spikes and errors helps fix real problems rather than just lowering load or ignoring issues.

3. Final Answer:

   Investigate causes of high peak response times and errors under load -> Option B

4. Quick Check:

   High peaks and errors = investigate and fix [OK]

Hint: Focus on fixing spikes and errors, not just lowering load [OK]

Common Mistakes:

• Ignoring error rate because average looks good
• Reducing users instead of fixing root causes
• Extending test time without addressing issues