Why CI integration enables continuous quality in PyTest - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does CI integration improve software quality?

Continuous Integration (CI) tools run tests automatically when code changes are made. Which of the following best explains why this helps maintain continuous quality?

A CI delays testing until deployment to production to save time.

B CI replaces manual testing, so testers are no longer needed.

C CI runs tests only after all features are complete, ensuring final quality.

D CI runs tests frequently, catching bugs early and preventing broken code from merging.

💡 Hint

Think about how often tests run and when bugs are found.

CI runs tests automatically on every code change, so bugs are found early. This prevents broken code from being merged and keeps quality high continuously.

❓ Predict Output

intermediate

What is the test result when CI runs this pytest test?

Given the following pytest test code, what will be the test result when run in a CI pipeline?

PyTest

def add(a, b):
    return a + b

def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0
    assert add(0, 0) == 0

A All tests pass successfully.

B One test fails due to incorrect assertion.

C SyntaxError occurs because of missing colon.

D TypeError occurs because add function is missing parameters.

💡 Hint

Check if the function and assertions are logically correct.

The add function correctly sums two numbers. All assertions match expected results, so all tests pass.

❓ assertion

advanced

Which assertion best verifies a web page title in CI tests?

In a CI test for a web app, you want to check that the page title is exactly 'Welcome Page'. Which pytest assertion is best?

PyTest

page_title = get_page_title()  # returns the current page title string

A assert 'Welcome' in page_title

B assert page_title == 'Welcome Page'

C assert page_title.startswith('Welcome')

D assert page_title != ''

💡 Hint

Think about exact match vs partial or presence checks.

To verify the title exactly matches 'Welcome Page', use equality assertion. Other options only check partial or non-empty strings.

🔧 Debug

advanced

Why does this pytest test fail in CI?

Review the pytest test below. It passes locally but fails in CI with a KeyError. Why?

PyTest

def get_user(data, key):
    return data[key]

def test_get_user():
    user = {'name': 'Alice'}
    assert get_user(user, 'name') == 'Alice'
    assert get_user(user, 'age') == 30

A KeyError occurs because 'age' key is missing in the dictionary.

B SyntaxError due to missing colon in function definition.

C AssertionError because 'age' value is 25, not 30.

D TypeError occurs because get_user expects a list, not dict.

💡 Hint

Check dictionary keys and what happens when a missing key is accessed.

Accessing a missing key in a dictionary raises KeyError. The test expects 'age' key which is not present, causing failure in CI.

❓ framework

expert

Which pytest fixture usage best supports CI continuous testing?

In a CI environment, you want to run tests that need a temporary database setup and teardown for each test. Which pytest fixture usage is best?

A

@pytest.fixture(scope='class')
def temp_db():
    setup_db()
    yield
    teardown_db()

B

@pytest.fixture(scope='session')
def temp_db():
    setup_db()
    yield
    teardown_db()

C

@pytest.fixture(scope='function')
def temp_db():
    setup_db()
    yield
    teardown_db()

D

@pytest.fixture(scope='module')
def temp_db():
    setup_db()
    yield
    teardown_db()

💡 Hint

Consider how often the database should reset for test isolation in CI.

Using function scope runs setup and teardown for each test, ensuring clean state and preventing test interference, which is ideal for CI.

Practice

1. What is the main benefit of integrating pytest with Continuous Integration (CI) systems?

easy

A. CI integration slows down the development process

B. Tests run only when manually triggered by developers

C. Tests run automatically on every code change to catch bugs early

D. CI integration replaces the need for writing tests

Solution

1. Step 1: Understand CI integration purpose

   CI systems run tests automatically whenever code changes are pushed.

2. Step 2: Identify the benefit of automatic testing

   This automatic testing helps catch bugs early and maintain software quality continuously.

3. Final Answer:

   Tests run automatically on every code change to catch bugs early -> Option C

4. Quick Check:

   CI runs tests automatically = A [OK]

Hint: CI runs tests on every change to catch bugs early [OK]

Common Mistakes:

• Thinking tests run only manually
• Believing CI slows development
• Assuming CI replaces writing tests

2. Which pytest command is commonly used in a CI pipeline to run all tests?

easy

A. pytest --run-all

B. pytest

C. pytest --ci-mode

D. pytest --skip

Solution

1. Step 1: Recall pytest basic command

   The basic command to run all tests is simply pytest.

2. Step 2: Check other options for validity

   Options like --run-all, --ci-mode, and --skip are not standard pytest commands.

3. Final Answer:

   pytest -> Option B

4. Quick Check:

   Run all tests = pytest [OK]

Hint: Use plain 'pytest' to run all tests in CI [OK]

Common Mistakes:

• Adding non-existent flags
• Using commands that skip tests
• Confusing pytest options with other tools

3. Given this pytest output in a CI pipeline:

============================= test session starts =============================
collected 3 items

test_sample.py ..F                                                      [100%]

================================== FAILURES ===================================
____________________________ test_divide_by_zero _____________________________

    def test_divide_by_zero():
>       assert 1 / 0
E       ZeroDivisionError: division by zero

What does this output tell you about the test results?

medium

A. All tests passed successfully

B. Tests did not run because of a syntax error

C. Tests were skipped

D. One test failed due to a division by zero error

Solution

1. Step 1: Analyze the test summary

   The output shows 3 tests collected, with two dots (.) meaning passed tests and one F meaning a failure.

2. Step 2: Identify failure cause

   The failure is due to a ZeroDivisionError in test_divide_by_zero.

3. Final Answer:

   One test failed due to a division by zero error -> Option D

4. Quick Check:

   F indicates one test failed due to ZeroDivisionError = C [OK]

Hint: F means failure; check error message for cause [OK]

Common Mistakes:

• Assuming all tests passed
• Confusing failure with skipped tests
• Ignoring error details

4. You added pytest tests to your project and integrated them with CI. However, the CI pipeline always shows zero tests collected. What is the most likely cause?

medium

A. Test files or functions are not named correctly (e.g., missing 'test_' prefix)

B. The CI server is offline

C. pytest is not installed on the CI server

D. Tests contain assertion errors

Solution

1. Step 1: Understand pytest test discovery rules

   pytest only collects tests from files and functions named starting with test_.

2. Step 2: Identify why zero tests are collected

   If no tests are found, likely the naming conventions are not followed, so pytest skips them.

3. Final Answer:

   Test files or functions are not named correctly (e.g., missing 'test_' prefix) -> Option A

4. Quick Check:

   pytest needs 'test_' prefix to find tests [OK]

Hint: Name test files/functions starting with 'test_' [OK]

Common Mistakes:

• Assuming CI server offline causes zero tests
• Ignoring pytest naming conventions
• Thinking assertion errors prevent test collection

5. In a CI pipeline using pytest, you want to ensure that tests run only if code formatting passes with black --check. Which approach best integrates this to maintain continuous quality?

hard

A. Run black --check first; if it fails, stop the pipeline; else run pytest tests

B. Run pytest tests first; then run black --check regardless of test results

C. Run both black --check and pytest tests in parallel without stopping

D. Skip black --check and only run pytest tests

Solution

1. Step 1: Understand quality gate concept in CI

   Code formatting checks should block further testing if they fail to maintain quality.

2. Step 2: Determine correct pipeline order

   Run black --check first; if it fails, stop pipeline to fix formatting before running tests.

3. Final Answer:

   Run black --check first; if it fails, stop the pipeline; else run pytest tests -> Option A

4. Quick Check:

   Fail fast on formatting, then test = A [OK]

Hint: Fail formatting check before tests to keep quality [OK]

Common Mistakes:

• Running tests before fixing formatting
• Ignoring formatting failures
• Running checks in parallel without order