Test metrics and KPIs in Testing Fundamentals - Build an Automation Script

Calculate and verify test metrics and KPIs for a test run

Preconditions (2)

Step 1: Count total number of test cases executed

Step 2: Count number of test cases passed

Step 3: Count number of test cases failed

Step 4: Count number of test cases blocked

Step 5: Calculate pass percentage = (passed / total) * 100

Step 6: Calculate fail percentage = (failed / total) * 100

Step 7: Calculate block percentage = (blocked / total) * 100

Step 8: Verify that the calculated percentages match expected values

✅ Expected Result: The calculated pass, fail, and block percentages are correct and match expected values

Automation Requirements - Python unittest

Assertions Needed:

Assert total test cases count is correct

Assert passed test cases count is correct

Assert failed test cases count is correct

Assert blocked test cases count is correct

Assert pass percentage calculation is correct

Assert fail percentage calculation is correct

Assert block percentage calculation is correct

Best Practices:

Use clear variable names for counts and percentages

Use setUp method to prepare test data

Use assertAlmostEqual for percentage comparisons with tolerance

Keep test logic simple and readable

Automated Solution

Testing Fundamentals

import unittest

class TestMetricsKPIs(unittest.TestCase):
    def setUp(self):
        # Sample test results data: list of statuses
        self.test_results = [
            'Passed', 'Failed', 'Passed', 'Blocked', 'Passed',
            'Failed', 'Passed', 'Passed', 'Blocked', 'Passed'
        ]
        self.total = len(self.test_results)

    def test_metrics_calculation(self):
        passed_count = self.test_results.count('Passed')
        failed_count = self.test_results.count('Failed')
        blocked_count = self.test_results.count('Blocked')

        pass_percentage = (passed_count / self.total) * 100
        fail_percentage = (failed_count / self.total) * 100
        block_percentage = (blocked_count / self.total) * 100

        # Assertions
        self.assertEqual(self.total, 10, "Total test cases should be 10")
        self.assertEqual(passed_count, 6, "Passed test cases should be 6")
        self.assertEqual(failed_count, 2, "Failed test cases should be 2")
        self.assertEqual(blocked_count, 2, "Blocked test cases should be 2")

        self.assertAlmostEqual(pass_percentage, 60.0, places=1, msg="Pass percentage should be 60.0")
        self.assertAlmostEqual(fail_percentage, 20.0, places=1, msg="Fail percentage should be 20.0")
        self.assertAlmostEqual(block_percentage, 20.0, places=1, msg="Block percentage should be 20.0")

if __name__ == '__main__':
    unittest.main()

The setUp method prepares the test data, a list of test case results with statuses.

We count how many test cases passed, failed, and were blocked using the count() method.

Then, we calculate the percentages by dividing each count by the total number of test cases and multiplying by 100.

Assertions check that counts and percentages match expected values. We use assertEqual for counts and assertAlmostEqual for percentages to allow small rounding differences.

This keeps the test clear, simple, and easy to maintain.

Common Mistakes - 3 Pitfalls

Hardcoding counts instead of calculating from data

Using assertEqual for floating point percentage comparisons

Not verifying total test case count

Bonus Challenge

Now add data-driven testing with 3 different test result sets and verify metrics for each

Show Hint