Coverage in CI pipelines in PyTest - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is code coverage in the context of CI pipelines?

Code coverage measures how much of your code is tested by automated tests during CI runs. It helps ensure tests cover important parts of the code.

beginner

Why integrate coverage reports in CI pipelines?

Integrating coverage reports in CI helps catch untested code early, improves code quality, and prevents bugs by enforcing testing standards automatically.

beginner

Which pytest plugin is commonly used to measure coverage?

The pytest-cov plugin is used to measure code coverage when running pytest tests. It generates coverage reports in various formats.

intermediate

How can you fail a CI build if coverage drops below a threshold using pytest?

Use pytest-cov options like --cov-fail-under=80 to fail the build if coverage is below 80%. This enforces minimum coverage standards automatically.

intermediate

What is a common format for coverage reports to integrate with CI tools?

The XML format (coverage.xml) is commonly used because many CI tools and coverage services can read it to display coverage results.

What does code coverage measure in CI pipelines?

A The percentage of code executed by tests

B The number of tests written

C The speed of test execution

D The number of bugs found

Code coverage measures how much of the code is executed by automated tests during CI runs.

Which pytest plugin helps generate coverage reports?

A pytest-xdist

B pytest-mock

C pytest-html

D pytest-cov

pytest-cov is the plugin used to measure and report code coverage.

How can you make a CI build fail if coverage is too low?

A Use --max-time option with pytest

B Use --fail-on-error option with pytest

C Use --cov-fail-under option with pytest-cov

D Use --skip-covered option with pytest-cov

The --cov-fail-under option sets a minimum coverage threshold that fails the build if not met.

Which coverage report format is widely supported by CI tools?

A XML

B TXT

C CSV

D JSON

XML format (coverage.xml) is commonly used for integration with CI tools and coverage services.

Why is it important to include coverage in CI pipelines?

A To speed up the build process

B To ensure tests cover critical code and maintain quality

C To reduce the number of tests

D To avoid writing tests

Coverage helps ensure tests cover important code, improving quality and catching bugs early.

Explain how to set up pytest coverage reporting in a CI pipeline.

Think about the commands and CI configuration steps.

Describe the benefits of enforcing coverage thresholds in CI pipelines.

Consider quality and team discipline reasons.