Test metrics and KPIs in Testing Fundamentals - Cheat Sheet & Quick Revision

Recall & Review

beginner

What are test metrics in software testing?

Test metrics are measurements used to assess the quality, progress, and effectiveness of testing activities. They help teams understand how well testing is going and where improvements are needed.

beginner

Define KPIs in the context of software testing.

KPIs (Key Performance Indicators) are specific, important metrics chosen to track the success of testing efforts against goals. They focus on critical areas like defect detection rate or test coverage.

beginner

Why is test coverage an important test metric?

Test coverage shows how much of the software's code or features have been tested. It helps ensure that important parts are not missed and reduces the risk of hidden bugs.

intermediate

Explain the difference between defect density and defect leakage.

Defect density measures the number of defects found per size of the software (like per 1000 lines of code). Defect leakage is the number of defects that escape testing and are found later by users.

intermediate

How can test metrics improve the testing process?

By tracking metrics, teams can spot problems early, measure progress, and make informed decisions to improve test planning, execution, and quality.

Which of the following is a Key Performance Indicator (KPI) in testing?

A Number of test cases executed

B Number of meetings held

C Defect detection percentage

D Lines of code written

Defect detection percentage is a KPI because it measures how effectively defects are found during testing.

What does test coverage measure?

A The number of defects fixed

B The number of testers on the team

C The time taken to write test cases

D The percentage of code or features tested

Test coverage measures how much of the software's code or features have been tested.

Defect leakage refers to:

A Defects found after release by users

B Defects found during testing

C Defects fixed by developers

D Defects ignored by testers

Defect leakage means defects that were missed during testing and found later by users.

Why are test metrics important?

A To measure and improve testing effectiveness

B To replace manual testing

C To increase the number of test cases

D To reduce the number of testers

Test metrics help measure progress and improve the quality and effectiveness of testing.

Which metric would best help track the speed of fixing defects?

A Test coverage

B Defect resolution time

C Defect density

D Number of test cases

Defect resolution time measures how quickly defects are fixed after being found.

Explain what test metrics and KPIs are and why they matter in software testing.

Think about how measuring helps improve any activity, like tracking your exercise progress.

Describe how defect density and defect leakage differ and why both are important to track.

Compare finding problems inside the factory versus problems customers find after buying.

Practice

1. Which of the following best describes a test metric in software testing?

easy

A. A tool used to automate tests

B. A measurable value that shows testing progress or quality

C. A list of test cases to be executed

D. A document describing software requirements

Solution

1. Step 1: Understand the definition of test metrics

   Test metrics are measurable values that help track testing progress or quality.

2. Step 2: Compare options with the definition

   Only A measurable value that shows testing progress or quality describes a measurable value related to testing progress or quality.

3. Final Answer:

   A measurable value that shows testing progress or quality -> Option B

4. Quick Check:

   Test metric = measurable value [OK]

Hint: Test metrics measure progress or quality, not tools or documents [OK]

Common Mistakes:

• Confusing test metrics with test cases
• Thinking test metrics are tools
• Mixing metrics with documentation

2. Which of the following is the correct formula for calculating Defect Density?

easy

A. Number of defects / Size of software module

B. Number of test cases executed / Total test cases

C. Total test time / Number of testers

D. Number of passed tests / Number of failed tests

Solution

1. Step 1: Recall Defect Density formula

   Defect Density is calculated as the number of defects divided by the size of the software module (e.g., lines of code).

2. Step 2: Match formula with options

   Number of defects / Size of software module matches the correct formula; others describe different metrics.

3. Final Answer:

   Number of defects / Size of software module -> Option A

4. Quick Check:

   Defect Density = defects / size [OK]

Hint: Defect Density = defects divided by size of module [OK]

Common Mistakes:

• Mixing defect density with test execution ratio
• Confusing numerator or denominator
• Using passed tests instead of defects

3. Given the following data:
Total test cases = 100
Test cases executed = 80
Test cases passed = 60
Defects found = 10
What is the Test Execution Rate (in percentage)?

medium

A. 90%

B. 60%

C. 75%

D. 80%

Solution

1. Step 1: Understand Test Execution Rate formula

   Test Execution Rate = (Test cases executed / Total test cases) * 100

2. Step 2: Calculate using given data

   (80 / 100) * 100 = 80%

3. Final Answer:

   80% -> Option D

4. Quick Check:

   Execution Rate = executed / total * 100 = 80% [OK]

Hint: Divide executed by total test cases, multiply by 100 [OK]

Common Mistakes:

• Using passed tests instead of executed tests
• Forgetting to multiply by 100 for percentage
• Mixing up numerator and denominator

4. A test report shows:
Total tests = 50
Passed tests = 45
Failed tests = 10
Identify the error in the report.

medium

A. Total tests should equal passed plus failed tests

B. Failed tests cannot be more than passed tests

C. No error, the report is correct

D. Passed tests cannot be more than total tests

Solution

1. Step 1: Check sum of passed and failed tests

   Passed tests (45) + Failed tests (10) = 55, which is greater than total tests (50).

2. Step 2: Verify consistency of totals

   Total tests should equal the sum of passed and failed tests for accuracy.

3. Final Answer:

   Total tests should equal passed plus failed tests -> Option A

4. Quick Check:

   Total = passed + failed [OK]

Hint: Total tests must equal passed plus failed tests [OK]

Common Mistakes:

• Ignoring mismatch in totals
• Assuming passed tests can exceed total
• Not validating report consistency

5. You want to improve your testing process by focusing on the most important metrics. Which KPI should you track to measure the effectiveness of your testing in finding defects?

hard

A. Test Case Preparation Status

B. Test Execution Rate

C. Defect Detection Percentage (DDP)

D. Test Environment Availability

Solution

1. Step 1: Understand KPI purpose

   KPIs highlight key metrics to track for specific goals, here to measure defect finding effectiveness.

2. Step 2: Identify KPI related to defect detection

   Defect Detection Percentage (DDP) measures how many defects are found compared to total defects, showing testing effectiveness.

3. Step 3: Compare other options

   Test Execution Rate measures progress, Test Case Preparation Status and Environment Availability do not measure defect finding effectiveness.

4. Final Answer:

   Defect Detection Percentage (DDP) -> Option C

5. Quick Check:

   Effectiveness KPI = Defect Detection Percentage [OK]

Hint: Track Defect Detection Percentage to measure defect finding [OK]

Common Mistakes:

• Confusing progress metrics with effectiveness KPIs
• Choosing environment or preparation metrics
• Ignoring the goal of measuring defect detection