Bird
Raised Fist0
Testing Fundamentalstesting~15 mins

Test metrics and KPIs in Testing Fundamentals - Build an Automation Script

Choose your learning style10 modes available
Calculate and verify test metrics and KPIs for a test run
Preconditions (2)
Step 1: Count total number of test cases executed
Step 2: Count number of test cases passed
Step 3: Count number of test cases failed
Step 4: Count number of test cases blocked
Step 5: Calculate pass percentage = (passed / total) * 100
Step 6: Calculate fail percentage = (failed / total) * 100
Step 7: Calculate block percentage = (blocked / total) * 100
Step 8: Verify that the calculated percentages match expected values
✅ Expected Result: The calculated pass, fail, and block percentages are correct and match expected values
Automation Requirements - Python unittest
Assertions Needed:
Assert total test cases count is correct
Assert passed test cases count is correct
Assert failed test cases count is correct
Assert blocked test cases count is correct
Assert pass percentage calculation is correct
Assert fail percentage calculation is correct
Assert block percentage calculation is correct
Best Practices:
Use clear variable names for counts and percentages
Use setUp method to prepare test data
Use assertAlmostEqual for percentage comparisons with tolerance
Keep test logic simple and readable
Automated Solution
Testing Fundamentals
import unittest

class TestMetricsKPIs(unittest.TestCase):
    def setUp(self):
        # Sample test results data: list of statuses
        self.test_results = [
            'Passed', 'Failed', 'Passed', 'Blocked', 'Passed',
            'Failed', 'Passed', 'Passed', 'Blocked', 'Passed'
        ]
        self.total = len(self.test_results)

    def test_metrics_calculation(self):
        passed_count = self.test_results.count('Passed')
        failed_count = self.test_results.count('Failed')
        blocked_count = self.test_results.count('Blocked')

        pass_percentage = (passed_count / self.total) * 100
        fail_percentage = (failed_count / self.total) * 100
        block_percentage = (blocked_count / self.total) * 100

        # Assertions
        self.assertEqual(self.total, 10, "Total test cases should be 10")
        self.assertEqual(passed_count, 6, "Passed test cases should be 6")
        self.assertEqual(failed_count, 2, "Failed test cases should be 2")
        self.assertEqual(blocked_count, 2, "Blocked test cases should be 2")

        self.assertAlmostEqual(pass_percentage, 60.0, places=1, msg="Pass percentage should be 60.0")
        self.assertAlmostEqual(fail_percentage, 20.0, places=1, msg="Fail percentage should be 20.0")
        self.assertAlmostEqual(block_percentage, 20.0, places=1, msg="Block percentage should be 20.0")

if __name__ == '__main__':
    unittest.main()

The setUp method prepares the test data, a list of test case results with statuses.

We count how many test cases passed, failed, and were blocked using the count() method.

Then, we calculate the percentages by dividing each count by the total number of test cases and multiplying by 100.

Assertions check that counts and percentages match expected values. We use assertEqual for counts and assertAlmostEqual for percentages to allow small rounding differences.

This keeps the test clear, simple, and easy to maintain.

Common Mistakes - 3 Pitfalls
Hardcoding counts instead of calculating from data
Using assertEqual for floating point percentage comparisons
Not verifying total test case count
Bonus Challenge

Now add data-driven testing with 3 different test result sets and verify metrics for each

Show Hint

Practice

(1/5)
1. Which of the following best describes a test metric in software testing?
easy
A. A tool used to automate tests
B. A measurable value that shows testing progress or quality
C. A list of test cases to be executed
D. A document describing software requirements

Solution

  1. Step 1: Understand the definition of test metrics

    Test metrics are measurable values that help track testing progress or quality.
  2. Step 2: Compare options with the definition

    Only A measurable value that shows testing progress or quality describes a measurable value related to testing progress or quality.
  3. Final Answer:

    A measurable value that shows testing progress or quality -> Option B
  4. Quick Check:

    Test metric = measurable value [OK]
Hint: Test metrics measure progress or quality, not tools or documents [OK]
Common Mistakes:
  • Confusing test metrics with test cases
  • Thinking test metrics are tools
  • Mixing metrics with documentation
2. Which of the following is the correct formula for calculating Defect Density?
easy
A. Number of defects / Size of software module
B. Number of test cases executed / Total test cases
C. Total test time / Number of testers
D. Number of passed tests / Number of failed tests

Solution

  1. Step 1: Recall Defect Density formula

    Defect Density is calculated as the number of defects divided by the size of the software module (e.g., lines of code).
  2. Step 2: Match formula with options

    Number of defects / Size of software module matches the correct formula; others describe different metrics.
  3. Final Answer:

    Number of defects / Size of software module -> Option A
  4. Quick Check:

    Defect Density = defects / size [OK]
Hint: Defect Density = defects divided by size of module [OK]
Common Mistakes:
  • Mixing defect density with test execution ratio
  • Confusing numerator or denominator
  • Using passed tests instead of defects
3. Given the following data:
Total test cases = 100
Test cases executed = 80
Test cases passed = 60
Defects found = 10

What is the Test Execution Rate (in percentage)?
medium
A. 90%
B. 60%
C. 75%
D. 80%

Solution

  1. Step 1: Understand Test Execution Rate formula

    Test Execution Rate = (Test cases executed / Total test cases) * 100
  2. Step 2: Calculate using given data

    (80 / 100) * 100 = 80%
  3. Final Answer:

    80% -> Option D
  4. Quick Check:

    Execution Rate = executed / total * 100 = 80% [OK]
Hint: Divide executed by total test cases, multiply by 100 [OK]
Common Mistakes:
  • Using passed tests instead of executed tests
  • Forgetting to multiply by 100 for percentage
  • Mixing up numerator and denominator
4. A test report shows:
Total tests = 50
Passed tests = 45
Failed tests = 10

Identify the error in the report.
medium
A. Total tests should equal passed plus failed tests
B. Failed tests cannot be more than passed tests
C. No error, the report is correct
D. Passed tests cannot be more than total tests

Solution

  1. Step 1: Check sum of passed and failed tests

    Passed tests (45) + Failed tests (10) = 55, which is greater than total tests (50).
  2. Step 2: Verify consistency of totals

    Total tests should equal the sum of passed and failed tests for accuracy.
  3. Final Answer:

    Total tests should equal passed plus failed tests -> Option A
  4. Quick Check:

    Total = passed + failed [OK]
Hint: Total tests must equal passed plus failed tests [OK]
Common Mistakes:
  • Ignoring mismatch in totals
  • Assuming passed tests can exceed total
  • Not validating report consistency
5. You want to improve your testing process by focusing on the most important metrics. Which KPI should you track to measure the effectiveness of your testing in finding defects?
hard
A. Test Case Preparation Status
B. Test Execution Rate
C. Defect Detection Percentage (DDP)
D. Test Environment Availability

Solution

  1. Step 1: Understand KPI purpose

    KPIs highlight key metrics to track for specific goals, here to measure defect finding effectiveness.
  2. Step 2: Identify KPI related to defect detection

    Defect Detection Percentage (DDP) measures how many defects are found compared to total defects, showing testing effectiveness.
  3. Step 3: Compare other options

    Test Execution Rate measures progress, Test Case Preparation Status and Environment Availability do not measure defect finding effectiveness.
  4. Final Answer:

    Defect Detection Percentage (DDP) -> Option C
  5. Quick Check:

    Effectiveness KPI = Defect Detection Percentage [OK]
Hint: Track Defect Detection Percentage to measure defect finding [OK]
Common Mistakes:
  • Confusing progress metrics with effectiveness KPIs
  • Choosing environment or preparation metrics
  • Ignoring the goal of measuring defect detection
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard