Test coverage metrics in Testing Fundamentals - Build an Automation Script

Verify test coverage metrics calculation for a simple function

Preconditions (2)

Step 1: Write tests to cover all branches of the 'calculate_discount' function

Step 2: Run the tests with coverage measurement enabled

Step 3: Check the coverage report for the 'discounts.py' file

Step 4: Verify that the coverage report shows 100% coverage for statements and branches

✅ Expected Result: The coverage report shows 100% statement and branch coverage for 'discounts.py'

Automation Requirements - pytest with coverage.py

Assertions Needed:

Assert that coverage report shows 100% statement coverage

Assert that coverage report shows 100% branch coverage

Best Practices:

Use pytest for writing tests

Use coverage.py to measure coverage

Write tests that cover all logical branches

Use assertions to verify coverage percentages programmatically

Automated Solution

Testing Fundamentals

import coverage
import pytest
from discounts import calculate_discount

cov = coverage.Coverage(branch=True, source=["discounts"])
cov.start()

# Test cases covering all branches

def test_no_discount():
    assert calculate_discount(50) == 50

def test_discount_10_percent():
    assert calculate_discount(150) == 135

def test_discount_20_percent():
    assert calculate_discount(300) == 240

cov.stop()
cov.save()

# Analyze coverage data
analysis = cov.analysis2('discounts.py')
statements, missing, excluded, missing_branches = analysis

# Assertions for coverage
assert len(missing) == 0, f"Missing statements: {missing}"
assert len(missing_branches) == 0, f"Missing branches: {missing_branches}"

print("All coverage assertions passed: 100% statements and branches covered.")

This script uses coverage.py to measure test coverage of the calculate_discount function in discounts.py.

First, it starts coverage measurement with branch coverage enabled.

Then, it defines three test functions to cover all logical branches of the discount calculation.

After running tests, it stops coverage and saves the data.

It analyzes coverage data for discounts.py to find missing statements and branches.

Assertions check that no statements or branches are missing, ensuring 100% coverage.

If assertions pass, it prints a confirmation message.

Common Mistakes - 4 Pitfalls

Not enabling branch coverage in coverage.py

Writing tests that cover only some branches

Checking coverage manually instead of programmatically

{'mistake': 'Not specifying the source module in coverage configuration', 'why_bad': 'Coverage may include unrelated files or miss the target file.', 'correct_approach': 'Specify source=["module_name"] when creating Coverage object.'}

Bonus Challenge

Now add data-driven testing with 3 different input amounts to cover all discount cases

Show Hint

Practice

1. What does test coverage metric measure in software testing?

easy

A. The time taken to run all tests

B. The number of bugs found during testing

C. The percentage of code or features tested compared to total code or features

D. The number of testers involved in the project

Solution

1. Step 1: Understand the purpose of test coverage

   Test coverage measures how much of the software is tested, usually as a percentage.

2. Step 2: Identify what is compared in coverage

   It compares tested code or features against the total code or features available.

3. Final Answer:

   The percentage of code or features tested compared to total code or features -> Option C

4. Quick Check:

   Test coverage = tested items / total items [OK]

Hint: Coverage shows tested parts over total parts [OK]

Common Mistakes:

• Confusing coverage with number of bugs
• Thinking coverage measures test duration
• Assuming coverage counts testers

2. Which of the following is the correct formula for calculating test coverage percentage?

easy

A. (Number of tested code lines / Total code lines) * 100

B. (Total tests / Number of testers) * 100

C. (Number of bugs found / Total tests) * 100

D. (Number of tests run / Total number of tests) * 100

Solution

1. Step 1: Identify what test coverage measures

   Test coverage measures how much code is tested, so it relates to tested code lines over total code lines.

2. Step 2: Match formula to coverage concept

   The formula is (tested code lines / total code lines) * 100 to get percentage coverage.

3. Final Answer:

   (Number of tested code lines / Total code lines) * 100 -> Option A

4. Quick Check:

   Coverage % = tested lines / total lines * 100 [OK]

Hint: Coverage = tested code lines divided by total lines [OK]

Common Mistakes:

• Using number of tests run instead of code lines
• Confusing bugs found with coverage
• Dividing by number of testers

3. Consider a software module with 200 lines of code. If 150 lines are executed during testing, what is the test coverage percentage?

medium

A. 85%

B. 75%

C. 65%

D. 70%

Solution

1. Step 1: Identify total and tested lines

   Total lines = 200, tested lines = 150.

2. Step 2: Calculate coverage percentage

   Coverage = (150 / 200) * 100 = 0.75 * 100 = 75%.

3. Final Answer:

   75% -> Option B

4. Quick Check:

   150/200 = 0.75 = 75% [OK]

Hint: Divide tested lines by total lines, multiply by 100 [OK]

Common Mistakes:

• Dividing total lines by tested lines
• Forgetting to multiply by 100
• Mixing up tested and total lines

4. A tester reports 90% test coverage, but some critical code paths are not tested. What is the likely issue?

medium

A. Coverage metric only measures quantity, not quality of tests

B. Tester calculated coverage incorrectly

C. Test coverage always guarantees all paths tested

D. The software has no untested code

Solution

1. Step 1: Understand what coverage measures

   Coverage measures how much code is tested but not which specific paths or quality of tests.

2. Step 2: Identify why critical paths might be missed

   High coverage can still miss important paths if tests do not cover all scenarios.

3. Final Answer:

   Coverage metric only measures quantity, not quality of tests -> Option A

4. Quick Check:

   Coverage % ≠ test quality [OK]

Hint: Coverage shows quantity, not test quality [OK]

Common Mistakes:

• Assuming coverage means all paths tested
• Blaming tester calculation without proof
• Ignoring untested critical paths

5. You have a project with 3 modules: Module A (100 lines, 80 tested), Module B (150 lines, 120 tested), Module C (50 lines, 40 tested). What is the overall test coverage percentage?

hard

A. 70%

B. 75%

C. 77.5%

D. 80%

Solution

1. Step 1: Calculate total lines and total tested lines

   Total lines = 100 + 150 + 50 = 300 lines. Tested lines = 80 + 120 + 40 = 240 lines.

2. Step 2: Calculate overall coverage percentage

   Coverage = (240 / 300) * 100 = 80%.

3. Step 3: Recalculate carefully

   Sum tested lines: 80 + 120 + 40 = 240. Sum total lines: 100 + 150 + 50 = 300. Coverage = (240/300)*100 = 80%.

4. Final Answer:

   80% -> Option D

5. Quick Check:

   Total tested / total lines = 240/300 = 80% [OK]

Hint: Sum tested and total lines, then divide and multiply by 100 [OK]

Common Mistakes:

• Averaging percentages instead of totals
• Adding percentages directly
• Mixing tested and total lines