Defect metrics in Testing Fundamentals - Framework Patterns

Framework Mode - Defect metrics

Folder Structure for Defect Metrics Tracking

  defect-metrics-project/
  ├── data/                  # Raw defect data files (CSV, JSON)
  ├── reports/               # Generated defect metric reports
  ├── src/                   # Source code for metric calculations
  │   ├── calculations/      # Functions to calculate metrics
  │   ├── utils/             # Helper functions (date parsing, formatting)
  │   └── main.py            # Entry point to run metrics
  ├── tests/                 # Unit tests for metric calculations
  ├── config/                # Configuration files for environments
  │   └── settings.yaml      # Thresholds, project info
  └── README.md              # Project overview and instructions
  

Test Framework Layers for Defect Metrics

• Data Layer: Stores raw defect data collected from bug tracking tools.
• Calculation Layer: Contains functions to compute metrics like defect density, defect leakage, and mean time to fix.
• Reporting Layer: Generates human-readable reports and charts from calculated metrics.
• Configuration Layer: Holds settings such as project phases, severity levels, and metric thresholds.
• Test Layer: Unit tests to verify metric calculations are correct and handle edge cases.

Configuration Patterns

Use a YAML or JSON file to store configurable parameters like:

• Project phases (e.g., Development, Testing, Production)
• Severity levels (e.g., Critical, Major, Minor)
• Thresholds for acceptable defect rates
• Data source paths or API endpoints

This allows easy updates without changing code. Load config at runtime to adapt metrics calculations.

Test Reporting and CI/CD Integration

Generate clear reports showing defect trends and key metrics in formats like HTML or PDF.

Integrate metric calculations into CI/CD pipelines to run after each test cycle or release.

Use automated notifications (email, Slack) to alert teams if defect metrics exceed thresholds.

Store historical reports for trend analysis and continuous improvement.

Best Practices for Defect Metrics Framework

• Keep data accurate and up-to-date: Metrics are only useful if defect data is reliable.
• Automate metric calculations: Avoid manual errors and save time by scripting all computations.
• Use clear definitions: Define defect types and severity consistently across the project.
• Visualize trends: Use charts to help teams quickly understand defect patterns.
• Integrate with existing tools: Connect with bug trackers and CI systems for seamless data flow.

Self Check

Where in this folder structure would you add a new function to calculate "Defect Removal Efficiency"?

Key Result

Organize defect metrics framework with clear layers for data, calculations, reporting, configuration, and tests to track software quality effectively.

Practice

1. What does defect density measure in software testing?

easy

A. Time taken to fix a defect

B. Total number of test cases executed

C. Number of defects per size of the software

D. Number of users reporting defects

Solution

1. Step 1: Understand defect density meaning

   Defect density is a metric that counts defects relative to software size, like per thousand lines of code.

2. Step 2: Compare options with definition

   Only Number of defects per size of the software matches this definition; others relate to different metrics.

3. Final Answer:

   Number of defects per size of the software -> Option C

4. Quick Check:

   Defect density = defects / software size [OK]

Hint: Defect density = defects divided by software size [OK]

Common Mistakes:

• Confusing defect density with total defects
• Mixing defect density with test execution count
• Thinking defect density measures fix time

2. Which of the following is the correct formula for calculating defect density?

easy

A. Defect Density = Total Defects / Software Size

B. Defect Density = Software Size / Total Defects

C. Defect Density = Total Defects x Software Size

D. Defect Density = Total Defects + Software Size

Solution

1. Step 1: Recall defect density formula

   Defect density is calculated by dividing total defects by software size.

2. Step 2: Match formula with options

   Defect Density = Total Defects / Software Size correctly shows division of defects by size; others use incorrect operations.

3. Final Answer:

   Defect Density = Total Defects / Software Size -> Option A

4. Quick Check:

   Defect density = defects ÷ size [OK]

Hint: Defect density divides defects by size, not multiply [OK]

Common Mistakes:

• Multiplying defects by size instead of dividing
• Reversing numerator and denominator
• Adding defects and size instead of dividing

3. A software module has 1200 lines of code and 24 reported defects. What is the defect density?

medium

A. 20 defects per 1000 lines

B. 50 defects per 1000 lines

C. 0.02 defects per line

D. 0.02 defects per 1000 lines

Solution

1. Step 1: Calculate defects per 1000 lines

   Defect density = (24 defects / 1200 lines) x 1000 = 20 defects per 1000 lines.

2. Step 2: Compare with options

   20 defects per 1000 lines matches 20 defects per 1000 lines; others are incorrect calculations or units.

3. Final Answer:

   20 defects per 1000 lines -> Option A

4. Quick Check:

   (24/1200)*1000 = 20 [OK]

Hint: Multiply defects/lines by 1000 for defect density [OK]

Common Mistakes:

• Not scaling defects per 1000 lines
• Confusing defects per line with per 1000 lines
• Incorrect division order

4. A defect report shows severity as 'High' but the status is marked 'Closed' without fixing. What is the likely issue?

medium

A. Defect density calculation error

B. Defect was fixed but not retested

C. Test cases are incomplete

D. Defect severity and status are inconsistent

Solution

1. Step 1: Analyze severity and status meaning

   High severity means serious defect; status 'Closed' usually means fixed or resolved.

2. Step 2: Identify inconsistency

   Closing a high severity defect without fixing is inconsistent and indicates a reporting or process error.

3. Final Answer:

   Defect severity and status are inconsistent -> Option D

4. Quick Check:

   High severity + Closed without fix = inconsistency [OK]

Hint: High severity defects should not be closed without fix [OK]

Common Mistakes:

• Assuming defect density relates to status
• Confusing test case completeness with defect status
• Thinking defect was fixed without evidence

5. You have two modules: Module A with 2000 lines and 40 defects, Module B with 1000 lines and 30 defects. Which module has better quality based on defect density?

hard

A. Module A has better quality because it has fewer defects

B. Module A has better quality because its defect density is lower

C. Both modules have the same quality

D. Module B has better quality because it has fewer lines of code

Solution

1. Step 1: Calculate defect density for Module A

   Module A defect density = 40 / 2000 = 0.02 defects per line.

2. Step 2: Calculate defect density for Module B

   Module B defect density = 30 / 1000 = 0.03 defects per line.

3. Step 3: Compare defect densities

   Lower defect density means better quality; Module A's 0.02 is less than Module B's 0.03.

4. Final Answer:

   Module A has better quality because its defect density is lower -> Option B

5. Quick Check:

   0.02 < 0.03 means Module A better [OK]

Hint: Lower defect density means better quality [OK]

Common Mistakes:

• Comparing only defect counts without size
• Assuming fewer lines means better quality
• Saying both have same quality without calculation