Defect metrics in Testing Fundamentals - Full Explanation

Introduction

When building software, it is important to know how many problems or bugs exist and how serious they are. Defect metrics help teams measure and understand these problems so they can improve the quality of their software.

Explanation

Defect Density

Defect density measures how many defects are found in a certain size of software, usually per thousand lines of code. It helps teams see how error-prone the software is compared to its size.

Defect density shows the number of defects relative to the size of the software.

Defect Severity

Defect severity describes how serious a defect is and how much it affects the software's functionality. High severity defects cause major problems, while low severity defects are minor issues.

Defect severity ranks defects by their impact on software use.

Defect Arrival Rate

Defect arrival rate tracks how many defects are reported over time during testing or after release. It helps teams understand when most defects appear and if the software is improving.

Defect arrival rate shows how defects are discovered over time.

Defect Resolution Time

Defect resolution time measures how long it takes to fix a defect from the moment it is reported. Shorter times mean faster problem solving and better team efficiency.

Defect resolution time tracks how quickly defects are fixed.

Defect Leakage

Defect leakage counts defects that were not found during testing but were discovered by users after release. It indicates gaps in the testing process.

Defect leakage reveals defects missed before software release.

Real World Analogy

Imagine a car factory checking cars for problems before selling them. They count how many cars have issues, how serious the issues are, how fast they find and fix problems, and if any problems are found after the cars are sold.

Defect Density → Number of cars with problems compared to total cars made

Defect Severity → How serious each car problem is, like a flat tire versus engine failure

Defect Arrival Rate → How many car problems are found each day during inspections

Defect Resolution Time → How long it takes mechanics to fix each car problem

Defect Leakage → Problems found by customers after buying the car that were missed in inspections

Diagram

Diagram

┌─────────────────────┐
│     Defect Metrics   │
├─────────┬───────────┤
│ Density │ Severity  │
├─────────┼───────────┤
│ Arrival │ Resolution│
│ Rate    │ Time      │
├─────────┴───────────┤
│      Leakage        │
└─────────────────────┘

This diagram shows the main defect metrics grouped together to measure software quality.

Key Facts

Defect Density → Number of defects found per thousand lines of code.

Defect Severity → Classification of defects based on their impact on software.

Defect Arrival Rate → Number of defects reported over a period of time.

Defect Resolution Time → Time taken to fix a defect after it is reported.

Defect Leakage → Defects found by users after software release that were missed in testing.

Common Confusions

Believing defect density alone shows software quality.

Believing defect density alone shows software quality. Defect density must be combined with severity and resolution time to fully understand software quality.

Assuming all defects have the same impact.

Assuming all defects have the same impact. Defect severity shows that some defects cause bigger problems than others.

Thinking defect leakage means testing failed completely.

Thinking defect leakage means testing failed completely. Some defects are very hard to find before release; leakage highlights areas to improve testing, not total failure.

Summary

Defect metrics help teams measure and improve software quality by tracking problems found and fixed.

Key metrics include defect density, severity, arrival rate, resolution time, and leakage.

Understanding these metrics together gives a clear picture of software health and testing effectiveness.

Practice

1. What does defect density measure in software testing?

easy

A. Time taken to fix a defect

B. Total number of test cases executed

C. Number of defects per size of the software

D. Number of users reporting defects

Solution

1. Step 1: Understand defect density meaning

   Defect density is a metric that counts defects relative to software size, like per thousand lines of code.

2. Step 2: Compare options with definition

   Only Number of defects per size of the software matches this definition; others relate to different metrics.

3. Final Answer:

   Number of defects per size of the software -> Option C

4. Quick Check:

   Defect density = defects / software size [OK]

Hint: Defect density = defects divided by software size [OK]

Common Mistakes:

• Confusing defect density with total defects
• Mixing defect density with test execution count
• Thinking defect density measures fix time

2. Which of the following is the correct formula for calculating defect density?

easy

A. Defect Density = Total Defects / Software Size

B. Defect Density = Software Size / Total Defects

C. Defect Density = Total Defects x Software Size

D. Defect Density = Total Defects + Software Size

Solution

1. Step 1: Recall defect density formula

   Defect density is calculated by dividing total defects by software size.

2. Step 2: Match formula with options

   Defect Density = Total Defects / Software Size correctly shows division of defects by size; others use incorrect operations.

3. Final Answer:

   Defect Density = Total Defects / Software Size -> Option A

4. Quick Check:

   Defect density = defects ÷ size [OK]

Hint: Defect density divides defects by size, not multiply [OK]

Common Mistakes:

• Multiplying defects by size instead of dividing
• Reversing numerator and denominator
• Adding defects and size instead of dividing

3. A software module has 1200 lines of code and 24 reported defects. What is the defect density?

medium

A. 20 defects per 1000 lines

B. 50 defects per 1000 lines

C. 0.02 defects per line

D. 0.02 defects per 1000 lines

Solution

1. Step 1: Calculate defects per 1000 lines

   Defect density = (24 defects / 1200 lines) x 1000 = 20 defects per 1000 lines.

2. Step 2: Compare with options

   20 defects per 1000 lines matches 20 defects per 1000 lines; others are incorrect calculations or units.

3. Final Answer:

   20 defects per 1000 lines -> Option A

4. Quick Check:

   (24/1200)*1000 = 20 [OK]

Hint: Multiply defects/lines by 1000 for defect density [OK]

Common Mistakes:

• Not scaling defects per 1000 lines
• Confusing defects per line with per 1000 lines
• Incorrect division order

4. A defect report shows severity as 'High' but the status is marked 'Closed' without fixing. What is the likely issue?

medium

A. Defect density calculation error

B. Defect was fixed but not retested

C. Test cases are incomplete

D. Defect severity and status are inconsistent

Solution

1. Step 1: Analyze severity and status meaning

   High severity means serious defect; status 'Closed' usually means fixed or resolved.

2. Step 2: Identify inconsistency

   Closing a high severity defect without fixing is inconsistent and indicates a reporting or process error.

3. Final Answer:

   Defect severity and status are inconsistent -> Option D

4. Quick Check:

   High severity + Closed without fix = inconsistency [OK]

Hint: High severity defects should not be closed without fix [OK]

Common Mistakes:

• Assuming defect density relates to status
• Confusing test case completeness with defect status
• Thinking defect was fixed without evidence

5. You have two modules: Module A with 2000 lines and 40 defects, Module B with 1000 lines and 30 defects. Which module has better quality based on defect density?

hard

A. Module A has better quality because it has fewer defects

B. Module A has better quality because its defect density is lower

C. Both modules have the same quality

D. Module B has better quality because it has fewer lines of code

Solution

1. Step 1: Calculate defect density for Module A

   Module A defect density = 40 / 2000 = 0.02 defects per line.

2. Step 2: Calculate defect density for Module B

   Module B defect density = 30 / 1000 = 0.03 defects per line.

3. Step 3: Compare defect densities

   Lower defect density means better quality; Module A's 0.02 is less than Module B's 0.03.

4. Final Answer:

   Module A has better quality because its defect density is lower -> Option B

5. Quick Check:

   0.02 < 0.03 means Module A better [OK]

Hint: Lower defect density means better quality [OK]

Common Mistakes:

• Comparing only defect counts without size
• Assuming fewer lines means better quality
• Saying both have same quality without calculation