Test coverage metrics in Testing Fundamentals - Test Execution Trace

Test Overview

This test checks if the test coverage tool correctly measures the percentage of code lines executed during testing. It verifies that the coverage report matches the expected coverage value.

Test Code - unittest with coverage.py

Testing Fundamentals

import unittest
import coverage

class Calculator:
    def add(self, a, b):
        return a + b
    def subtract(self, a, b):
        return a - b

class TestCalculator(unittest.TestCase):
    def test_add(self):
        calc = Calculator()
        self.assertEqual(calc.add(2, 3), 5)

if __name__ == '__main__':
    cov = coverage.Coverage()
    cov.start()
    unittest.main(exit=False)
    cov.stop()
    cov.save()
    analysis = cov.analysis('test_coverage_metrics.py')
    coverage_percent = analysis[4]
    print(f"Coverage: {coverage_percent:.2f}%")
    assert coverage_percent >= 50, "Coverage is less than 50%"

Execution Trace - 4 Steps

Step	Action	System State	Assertion	Result
1	Test starts and coverage measurement begins	Coverage tool initialized, test runner ready	—	PASS
2	Test runner executes test_add method of TestCalculator	Calculator instance created, add method called with (2,3)	Check if add(2,3) returns 5	PASS
3	Coverage measurement stops and data saved	Coverage data collected for executed lines	—	PASS
4	Coverage report generated and coverage percentage calculated	Coverage report shows lines executed vs total lines	Assert coverage_percent >= 50	PASS

Failure Scenario

Failing Condition: Test coverage is below 50% because some code lines are not executed

Execution Trace Quiz - 3 Questions

What does the coverage tool measure in this test?

A The time taken to run the tests

B The percentage of code lines executed during the test

C The number of tests passed

D The number of errors in the code

Key Result

Always write tests that cover important code paths to ensure high test coverage and catch potential bugs early.

Practice

1. What does test coverage metric measure in software testing?

easy

A. The time taken to run all tests

B. The number of bugs found during testing

C. The percentage of code or features tested compared to total code or features

D. The number of testers involved in the project

Solution

1. Step 1: Understand the purpose of test coverage

   Test coverage measures how much of the software is tested, usually as a percentage.

2. Step 2: Identify what is compared in coverage

   It compares tested code or features against the total code or features available.

3. Final Answer:

   The percentage of code or features tested compared to total code or features -> Option C

4. Quick Check:

   Test coverage = tested items / total items [OK]

Hint: Coverage shows tested parts over total parts [OK]

Common Mistakes:

• Confusing coverage with number of bugs
• Thinking coverage measures test duration
• Assuming coverage counts testers

2. Which of the following is the correct formula for calculating test coverage percentage?

easy

A. (Number of tested code lines / Total code lines) * 100

B. (Total tests / Number of testers) * 100

C. (Number of bugs found / Total tests) * 100

D. (Number of tests run / Total number of tests) * 100

Solution

1. Step 1: Identify what test coverage measures

   Test coverage measures how much code is tested, so it relates to tested code lines over total code lines.

2. Step 2: Match formula to coverage concept

   The formula is (tested code lines / total code lines) * 100 to get percentage coverage.

3. Final Answer:

   (Number of tested code lines / Total code lines) * 100 -> Option A

4. Quick Check:

   Coverage % = tested lines / total lines * 100 [OK]

Hint: Coverage = tested code lines divided by total lines [OK]

Common Mistakes:

• Using number of tests run instead of code lines
• Confusing bugs found with coverage
• Dividing by number of testers

3. Consider a software module with 200 lines of code. If 150 lines are executed during testing, what is the test coverage percentage?

medium

A. 85%

B. 75%

C. 65%

D. 70%

Solution

1. Step 1: Identify total and tested lines

   Total lines = 200, tested lines = 150.

2. Step 2: Calculate coverage percentage

   Coverage = (150 / 200) * 100 = 0.75 * 100 = 75%.

3. Final Answer:

   75% -> Option B

4. Quick Check:

   150/200 = 0.75 = 75% [OK]

Hint: Divide tested lines by total lines, multiply by 100 [OK]

Common Mistakes:

• Dividing total lines by tested lines
• Forgetting to multiply by 100
• Mixing up tested and total lines

4. A tester reports 90% test coverage, but some critical code paths are not tested. What is the likely issue?

medium

A. Coverage metric only measures quantity, not quality of tests

B. Tester calculated coverage incorrectly

C. Test coverage always guarantees all paths tested

D. The software has no untested code

Solution

1. Step 1: Understand what coverage measures

   Coverage measures how much code is tested but not which specific paths or quality of tests.

2. Step 2: Identify why critical paths might be missed

   High coverage can still miss important paths if tests do not cover all scenarios.

3. Final Answer:

   Coverage metric only measures quantity, not quality of tests -> Option A

4. Quick Check:

   Coverage % ≠ test quality [OK]

Hint: Coverage shows quantity, not test quality [OK]

Common Mistakes:

• Assuming coverage means all paths tested
• Blaming tester calculation without proof
• Ignoring untested critical paths

5. You have a project with 3 modules: Module A (100 lines, 80 tested), Module B (150 lines, 120 tested), Module C (50 lines, 40 tested). What is the overall test coverage percentage?

hard

A. 70%

B. 75%

C. 77.5%

D. 80%

Solution

1. Step 1: Calculate total lines and total tested lines

   Total lines = 100 + 150 + 50 = 300 lines. Tested lines = 80 + 120 + 40 = 240 lines.

2. Step 2: Calculate overall coverage percentage

   Coverage = (240 / 300) * 100 = 80%.

3. Step 3: Recalculate carefully

   Sum tested lines: 80 + 120 + 40 = 240. Sum total lines: 100 + 150 + 50 = 300. Coverage = (240/300)*100 = 80%.

4. Final Answer:

   80% -> Option D

5. Quick Check:

   Total tested / total lines = 240/300 = 80% [OK]

Hint: Sum tested and total lines, then divide and multiply by 100 [OK]

Common Mistakes:

• Averaging percentages instead of totals
• Adding percentages directly
• Mixing tested and total lines