Performance test reporting in Testing Fundamentals - Build an Automation Script
import os import time import pytest REPORT_PATH = 'performance_report.html' # Simulate running load test and generating report # In real scenario, this would trigger Locust or other tool def run_load_test(): # Simulate test duration time.sleep(5) # wait 5 seconds to simulate test # Simulate report generation with open(REPORT_PATH, 'w') as f: f.write('<html><body>') f.write('<h1>Performance Test Report</h1>') f.write('<p>Average Response Time: 200ms</p>') f.write('<p>Throughput: 50 requests/sec</p>') f.write('<p>Error Rate: 0.5%</p>') f.write('<p>Test Duration: 5 minutes</p>') f.write('</body></html>') def test_performance_report_generation(): # Run the load test simulation run_load_test() # Wait for report file to exist (explicit wait) timeout = 10 poll_interval = 1 waited = 0 while not os.path.exists(REPORT_PATH) and waited < timeout: time.sleep(poll_interval) waited += poll_interval # Assert report file exists assert os.path.exists(REPORT_PATH), 'Report file was not generated' # Read report content with open(REPORT_PATH, 'r') as f: content = f.read() # Assert key metrics present assert 'Average Response Time' in content, 'Average Response Time metric missing' assert 'Throughput' in content, 'Throughput metric missing' assert 'Error Rate' in content, 'Error Rate metric missing' assert 'Test Duration' in content, 'Test Duration metric missing' # Extract error rate value import re match = re.search(r'Error Rate: ([0-9.]+)%', content) assert match is not None, 'Error Rate value not found' error_rate = float(match.group(1)) # Assert error rate below 1% assert error_rate < 1.0, f'Error rate too high: {error_rate}%' # Cleanup os.remove(REPORT_PATH)
This test script simulates running a performance load test and generating an HTML report.
The run_load_test() function waits 5 seconds to mimic test duration and creates a simple HTML report file with key metrics.
The test test_performance_report_generation() waits explicitly for the report file to appear, then reads its content.
It asserts the presence of average response time, throughput, error rate, and test duration in the report.
Using a regular expression, it extracts the error rate value and asserts it is below 1%.
Finally, it cleans up by deleting the report file.
This approach uses explicit waits, file checks, content parsing, and pytest assertions to verify performance test reporting.
Now add data-driven testing with 3 different load levels: 50, 100, and 200 virtual users
Practice
performance test report?Solution
Step 1: Understand the role of performance testing
Performance testing checks how software works when many users or tasks run at once.Step 2: Identify what the report shows
The report summarizes results like speed, response time, and stability under load.Final Answer:
To show how software behaves under different loads -> Option DQuick Check:
Performance test report = software behavior under load [OK]
- Confusing performance reports with bug reports
- Thinking reports describe UI or installation
- Ignoring load and speed details
Solution
Step 1: Review common report contents
Performance reports usually have graphs, user counts, and conclusions about performance.Step 2: Identify unrelated content
Steps to reproduce UI bugs belong in functional or bug reports, not performance reports.Final Answer:
Steps to reproduce a UI bug -> Option AQuick Check:
UI bug steps ≠ performance report content [OK]
- Including bug reproduction steps in performance reports
- Confusing test conclusion with bug details
- Ignoring graphs and user counts
Average response time: 2.5 seconds Peak users: 1000 Error rate: 0.5%
What does the error rate tell us?
Solution
Step 1: Understand error rate meaning
Error rate shows how many requests or actions failed compared to total attempts.Step 2: Match error rate to options
It is a percentage of failures, not time or user count.Final Answer:
The percentage of requests that failed during the test -> Option AQuick Check:
Error rate = % of failed requests [OK]
- Confusing error rate with response time
- Mixing error rate with user count
- Thinking error rate is max response time
Solution
Step 1: Analyze test load impact
Using only 10 users instead of 1000 means the load is too small to simulate real conditions.Step 2: Understand effect on results
Low load can hide performance problems or give misleading response times.Final Answer:
Test load is too low to reflect real performance -> Option CQuick Check:
Low user load = unreliable performance data [OK]
- Assuming response times always increase
- Ignoring incorrect user load
- Expecting zero error rate always
Average response time: 1.2s Peak response time: 5.0s Error rate: 3% Concurrent users: 500
Which action should you recommend to improve the system?
Solution
Step 1: Analyze report data
Peak response time is much higher than average, and error rate is 3%, which is significant under 500 users.Step 2: Decide best improvement action
Investigating causes of spikes and errors helps fix real problems rather than just lowering load or ignoring issues.Final Answer:
Investigate causes of high peak response times and errors under load -> Option BQuick Check:
High peaks and errors = investigate and fix [OK]
- Ignoring error rate because average looks good
- Reducing users instead of fixing root causes
- Extending test time without addressing issues
Ready to prove it?
Pass the test and this pattern moves unknown
