Why coverage measures test completeness in PyTest - Quick Recap

Recall & Review

beginner

What does test coverage measure in software testing?

Test coverage measures how much of the code is executed by the tests. It shows which parts of the code have been tested and which parts have not.

beginner

Why is high test coverage important?

High test coverage means more code is tested, reducing the chance of bugs. It helps ensure the software works as expected in many situations.

intermediate

Can 100% test coverage guarantee bug-free software?

No, 100% coverage means all code lines run during tests, but it does not guarantee all bugs are found. Tests must also check for correct behavior.

beginner

How does pytest help measure test coverage?

Pytest can be combined with coverage.py to track which lines of code run during tests. It generates reports showing coverage percentage and uncovered code.

beginner

What is a practical way to improve test coverage?

Add tests for parts of the code that are not covered yet. Focus on important or complex code to catch more bugs and improve software quality.

What does test coverage show?

A How fast tests run

B How many bugs are in the code

C Which parts of code are executed by tests

D How many users use the software

Test coverage shows which parts of the code are run during testing, helping measure test completeness.

Which tool can be used with pytest to measure coverage?

A Docker

B Selenium

C Jenkins

D coverage.py

coverage.py is a tool that tracks code execution during tests and works well with pytest.

Does 100% test coverage mean no bugs remain?

A No, tests must also check correctness

B No, coverage measures test speed

C Yes, coverage tests all user cases

D Yes, all bugs are found

Even with full coverage, tests must verify correct behavior to find bugs.

Why improve test coverage?

A To make tests run slower

B To reduce untested code and catch more bugs

C To increase software size

D To avoid writing tests

Improving coverage reduces untested code, helping find bugs and improve quality.

What does low test coverage indicate?

A Many code parts are not tested

B Tests are very fast

C Software is bug-free

D All tests passed

Low coverage means many code lines are not executed by tests, risking undetected bugs.

Explain why test coverage is a useful measure of test completeness.

Think about how coverage relates to code execution and test thoroughness.

Describe how pytest and coverage.py work together to measure test coverage.

Consider the roles of both tools in testing and reporting.

Practice

1. What does test coverage measure in pytest?

easy

A. How much of the code is executed by tests

B. How many tests are written

C. How fast the tests run

D. How many errors tests find

Solution

1. Step 1: Understand the meaning of coverage

   Coverage shows which parts of the code are run when tests execute.

2. Step 2: Compare options to coverage definition

   Only How much of the code is executed by tests matches this meaning, others describe different test aspects.

3. Final Answer:

   How much of the code is executed by tests -> Option A

4. Quick Check:

   Coverage = executed code percentage [OK]

Hint: Coverage = code run by tests, not test count [OK]

Common Mistakes:

• Confusing coverage with number of tests
• Thinking coverage measures test speed
• Believing coverage counts errors found

2. Which pytest command correctly runs tests with coverage measurement?

easy

A. pytest --cover

B. pytest --coverage

C. pytest -cov-report

D. pytest --cov

Solution

1. Step 1: Recall pytest coverage plugin syntax

   The correct flag to measure coverage is '--cov'.

2. Step 2: Check options for correctness

   Only pytest --cov uses the exact correct flag '--cov'. Others are invalid or incomplete.

3. Final Answer:

   pytest --cov -> Option D

4. Quick Check:

   Use --cov to enable coverage [OK]

Hint: Use '--cov' flag to measure coverage in pytest [OK]

Common Mistakes:

• Using '--coverage' instead of '--cov'
• Mixing coverage report flags with coverage run flags
• Typing '--cover' which is invalid

3. Given this pytest coverage output:

Name          Stmts   Miss  Cover
my_module.py     10      2    80%

What does the 'Miss' number mean?

medium

A. Number of lines not executed by tests

B. Number of errors in code

C. Number of tests skipped

D. Number of tests that failed

Solution

1. Step 1: Understand coverage report columns

   'Miss' shows how many lines of code were not run by tests.

2. Step 2: Match 'Miss' meaning to options

   Number of lines not executed by tests correctly describes 'Miss' as unexecuted lines; others describe unrelated test results.

3. Final Answer:

   Number of lines not executed by tests -> Option A

4. Quick Check:

   Miss = untested lines count [OK]

Hint: 'Miss' means lines tests did not run [OK]

Common Mistakes:

• Thinking 'Miss' counts failed tests
• Confusing 'Miss' with skipped tests
• Assuming 'Miss' means code errors

4. You run pytest with coverage but get 0% coverage report. What is the most likely cause?

medium

A. Tests passed too quickly

B. Tests did not execute any code

C. Coverage plugin is not installed

D. Code has no functions

Solution

1. Step 1: Analyze 0% coverage meaning

   0% coverage means no code lines were run during tests.

2. Step 2: Evaluate causes

   If the coverage plugin is not installed, pytest will not measure coverage and may silently produce 0% coverage report or no coverage data.

3. Final Answer:

   Coverage plugin is not installed -> Option C

4. Quick Check:

   Missing plugin causes no coverage data [OK]

Hint: 0% coverage often means coverage plugin missing [OK]

Common Mistakes:

• Assuming plugin missing causes 0% without errors
• Thinking fast tests mean low coverage
• Believing code without functions can't be covered

5. You want to improve test completeness using coverage. Which approach is best?

hard

A. Write more tests without checking coverage

B. Add tests targeting uncovered code lines shown by coverage report

C. Ignore coverage and focus on test speed

D. Remove tests that run covered code

Solution

1. Step 1: Understand coverage report use

   Coverage shows which code lines lack tests, guiding where to add tests.

2. Step 2: Evaluate options for improving completeness

   Only Add tests targeting uncovered code lines shown by coverage report uses coverage data to add tests for uncovered code, improving completeness.

3. Final Answer:

   Add tests targeting uncovered code lines shown by coverage report -> Option B

4. Quick Check:

   Use coverage to find and test missing code [OK]

Hint: Add tests where coverage report shows gaps [OK]

Common Mistakes:

• Writing tests blindly without coverage info
• Ignoring coverage to focus on speed
• Removing tests that cover code