Test metrics and KPIs in Testing Fundamentals - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Defect Density

Which of the following best describes the defect density metric in software testing?

A The number of defects found divided by the size of the software module, usually measured in lines of code.

B The total number of test cases executed during a test cycle.

C The percentage of test cases that passed successfully.

D The average time taken to fix a defect after it is reported.

💡 Hint

Think about how defects relate to the size of the software.

Defect density measures how many defects exist per unit size of software, helping teams understand quality relative to code size.

❓ Predict Output

intermediate

Calculating Test Execution Rate

What is the output of the following Python code that calculates the test execution rate?

Testing Fundamentals

total_tests = 150
executed_tests = 120
execution_rate = (executed_tests / total_tests) * 100
print(f"Execution Rate: {execution_rate:.2f}%")

A Execution Rate: 80.00%

B Execution Rate: 100.00%

C Execution Rate: 120.00%

D Execution Rate: 75.00%

💡 Hint

Divide executed tests by total tests and multiply by 100.

The code calculates the percentage of tests executed out of total tests, which is (120/150)*100 = 80.00%.

❓ assertion

advanced

Validating Test Pass Rate Assertion

Given the following test results, which assertion correctly verifies that the pass rate is at least 90%?

total_tests = 200
passed_tests = 185
pass_rate = (passed_tests / total_tests) * 100

A assert pass_rate > 90, f"Pass rate too low: {pass_rate}%"

B assert pass_rate <= 90, f"Pass rate too low: {pass_rate}%"

C assert pass_rate == 90, f"Pass rate too low: {pass_rate}%"

D assert pass_rate >= 90, f"Pass rate too low: {pass_rate}%"

💡 Hint

Check if pass rate is at least 90% (greater than or equal).

Option D correctly asserts that pass_rate is 90% or more. Option D excludes exactly 90%, C requires exact 90%, and D is incorrect logic.

🔧 Debug

advanced

Debugging Incorrect Defect Leakage Calculation

The following code is intended to calculate defect leakage percentage but produces an incorrect result. What is the main issue?

defects_found_in_testing = 30
defects_found_in_production = 5
defect_leakage = defects_found_in_production / defects_found_in_testing * 100
print(f"Defect Leakage: {defect_leakage}%")

A Division should be reversed: defects_found_in_testing / defects_found_in_production.

B The calculation is correct; no issue present.

C Integer division is used; convert to float to avoid truncation.

D The variable names are swapped; defects_found_in_production should be numerator.

💡 Hint

Check the formula for defect leakage: defects in production divided by defects found in testing.

The formula is correct: defect leakage = (defects in production / defects found in testing) * 100. Python 3 uses float division by default, so no truncation occurs.

❓ framework

expert

Choosing the Best KPI for Release Readiness

As a QA lead, you want to decide if a software release is ready based on test metrics. Which KPI is the most reliable indicator to include in your report?

A Percentage of automated test cases executed successfully.

B Number of defects found during testing.

C Defect severity distribution and open critical defects count.

D Total number of test cases designed.

💡 Hint

Consider which metric reflects risk and quality impact most directly.

Defect severity distribution and count of open critical defects provide insight into risk and quality, making it the best KPI for release readiness.

Practice

1. Which of the following best describes a test metric in software testing?

easy

A. A tool used to automate tests

B. A measurable value that shows testing progress or quality

C. A list of test cases to be executed

D. A document describing software requirements

Solution

1. Step 1: Understand the definition of test metrics

   Test metrics are measurable values that help track testing progress or quality.

2. Step 2: Compare options with the definition

   Only A measurable value that shows testing progress or quality describes a measurable value related to testing progress or quality.

3. Final Answer:

   A measurable value that shows testing progress or quality -> Option B

4. Quick Check:

   Test metric = measurable value [OK]

Hint: Test metrics measure progress or quality, not tools or documents [OK]

Common Mistakes:

• Confusing test metrics with test cases
• Thinking test metrics are tools
• Mixing metrics with documentation

2. Which of the following is the correct formula for calculating Defect Density?

easy

A. Number of defects / Size of software module

B. Number of test cases executed / Total test cases

C. Total test time / Number of testers

D. Number of passed tests / Number of failed tests

Solution

1. Step 1: Recall Defect Density formula

   Defect Density is calculated as the number of defects divided by the size of the software module (e.g., lines of code).

2. Step 2: Match formula with options

   Number of defects / Size of software module matches the correct formula; others describe different metrics.

3. Final Answer:

   Number of defects / Size of software module -> Option A

4. Quick Check:

   Defect Density = defects / size [OK]

Hint: Defect Density = defects divided by size of module [OK]

Common Mistakes:

• Mixing defect density with test execution ratio
• Confusing numerator or denominator
• Using passed tests instead of defects

3. Given the following data:
Total test cases = 100
Test cases executed = 80
Test cases passed = 60
Defects found = 10
What is the Test Execution Rate (in percentage)?

medium

A. 90%

B. 60%

C. 75%

D. 80%

Solution

1. Step 1: Understand Test Execution Rate formula

   Test Execution Rate = (Test cases executed / Total test cases) * 100

2. Step 2: Calculate using given data

   (80 / 100) * 100 = 80%

3. Final Answer:

   80% -> Option D

4. Quick Check:

   Execution Rate = executed / total * 100 = 80% [OK]

Hint: Divide executed by total test cases, multiply by 100 [OK]

Common Mistakes:

• Using passed tests instead of executed tests
• Forgetting to multiply by 100 for percentage
• Mixing up numerator and denominator

4. A test report shows:
Total tests = 50
Passed tests = 45
Failed tests = 10
Identify the error in the report.

medium

A. Total tests should equal passed plus failed tests

B. Failed tests cannot be more than passed tests

C. No error, the report is correct

D. Passed tests cannot be more than total tests

Solution

1. Step 1: Check sum of passed and failed tests

   Passed tests (45) + Failed tests (10) = 55, which is greater than total tests (50).

2. Step 2: Verify consistency of totals

   Total tests should equal the sum of passed and failed tests for accuracy.

3. Final Answer:

   Total tests should equal passed plus failed tests -> Option A

4. Quick Check:

   Total = passed + failed [OK]

Hint: Total tests must equal passed plus failed tests [OK]

Common Mistakes:

• Ignoring mismatch in totals
• Assuming passed tests can exceed total
• Not validating report consistency

5. You want to improve your testing process by focusing on the most important metrics. Which KPI should you track to measure the effectiveness of your testing in finding defects?

hard

A. Test Case Preparation Status

B. Test Execution Rate

C. Defect Detection Percentage (DDP)

D. Test Environment Availability

Solution

1. Step 1: Understand KPI purpose

   KPIs highlight key metrics to track for specific goals, here to measure defect finding effectiveness.

2. Step 2: Identify KPI related to defect detection

   Defect Detection Percentage (DDP) measures how many defects are found compared to total defects, showing testing effectiveness.

3. Step 3: Compare other options

   Test Execution Rate measures progress, Test Case Preparation Status and Environment Availability do not measure defect finding effectiveness.

4. Final Answer:

   Defect Detection Percentage (DDP) -> Option C

5. Quick Check:

   Effectiveness KPI = Defect Detection Percentage [OK]

Hint: Track Defect Detection Percentage to measure defect finding [OK]

Common Mistakes:

• Confusing progress metrics with effectiveness KPIs
• Choosing environment or preparation metrics
• Ignoring the goal of measuring defect detection