Test coverage metrics in Testing Fundamentals - Full Explanation

Introduction

When writing tests for software, it can be hard to know if you have tested enough. Test coverage metrics help solve this by measuring how much of the code is actually checked by tests. This helps find gaps where bugs might hide.

Explanation

Line Coverage

Line coverage measures the percentage of code lines that have been executed by tests. If a line runs during testing, it counts as covered. This metric shows how much of the code is touched by tests but does not guarantee all logic is checked.

Line coverage shows what parts of the code have been run during tests.

Branch Coverage

Branch coverage checks if all possible paths from decision points like if-else statements have been tested. It ensures that both true and false outcomes of conditions are covered. This helps catch errors in different logical branches.

Branch coverage ensures all decision paths in the code are tested.

Function Coverage

Function coverage measures how many functions or methods have been called during testing. It helps confirm that all parts of the program’s structure are exercised by tests, but it does not check inside the functions.

Function coverage tracks which functions are tested.

Statement Coverage

Statement coverage is similar to line coverage but focuses on individual statements, including multiple statements on one line. It ensures every statement has been executed at least once during tests.

Statement coverage confirms every statement in the code runs during tests.

Limitations of Coverage Metrics

High coverage numbers do not guarantee bug-free code because tests might not check for correct behavior. Coverage only shows what code runs, not if tests are meaningful or if edge cases are covered.

Coverage metrics measure test reach but not test quality or correctness.

Real World Analogy

Imagine cleaning a house and wanting to know how much you cleaned. Line coverage is like checking which rooms you entered. Branch coverage is like checking if you cleaned both sides of a door. Function coverage is like checking if you cleaned every room. But just entering a room doesn’t mean you cleaned it well.

Line Coverage → Rooms entered during cleaning

Branch Coverage → Checking both sides of a door to clean

Function Coverage → Cleaning every room in the house

Statement Coverage → Cleaning every corner or spot in a room

Limitations of Coverage Metrics → Entering a room but not cleaning it properly

Diagram

Diagram

┌───────────────────────────────┐
│         Test Coverage          │
├───────────────┬───────────────┤
│ Line Coverage │ Branch Coverage│
│ (Lines run)   │ (Paths tested) │
├───────────────┼───────────────┤
│ Function Cover│ Statement Cover│
│ (Functions run)│ (Statements run)│
└───────────────┴───────────────┘
         ↓
  Limitations: Coverage ≠ Quality

Diagram showing different test coverage types and their relation, ending with a note on limitations.

Key Facts

Line Coverage → Measures the percentage of code lines executed by tests.

Branch Coverage → Measures if all decision paths in the code are tested.

Function Coverage → Measures how many functions are called during testing.

Statement Coverage → Measures if every statement in the code runs during tests.

Coverage Limitations → High coverage does not guarantee tests check for correct behavior.

Code Example

Testing Fundamentals

def is_even(n):
    if n % 2 == 0:
        return True
    else:
        return False

print(is_even(4))
print(is_even(5))

Common Confusions

High coverage means the code is bug-free.

High coverage means the code is bug-free. Coverage only shows which code runs during tests, not if tests verify correct results or cover all edge cases.

Line coverage and statement coverage are the same.

Line coverage and statement coverage are the same. Line coverage counts lines executed, but statement coverage counts each individual statement, which can be multiple per line.

Function coverage ensures all code inside functions is tested.

Function coverage ensures all code inside functions is tested. Function coverage only checks if functions are called, not if all code inside them runs.

Summary

Test coverage metrics measure how much of the code is executed during testing to find untested parts.

Different coverage types focus on lines, branches, functions, or statements to give a fuller picture of test reach.

Coverage numbers alone do not guarantee tests check for correct behavior or catch all bugs.