Why parametrize multiplies test coverage in PyTest - Challenge Your Understanding

Challenge - 5 Problems

🧠 Conceptual

intermediate

How does pytest parametrize increase test coverage?

Consider a test function that uses @pytest.mark.parametrize to run multiple input values. Why does this approach multiply test coverage?

A It skips tests that are redundant, reducing coverage but saving time.

B It runs tests in parallel, speeding up execution but not increasing coverage.

C It automatically generates new test functions from the code under test.

D It runs the same test logic multiple times with different inputs, covering more cases.

💡 Hint

Think about how running a test with different inputs affects the number of scenarios checked.

Parametrization runs the same test logic multiple times with different inputs. This means more scenarios are tested, increasing coverage.

❓ Predict Output

intermediate

Output of parametrized pytest test count

Given the following pytest test code, how many tests will be executed?

PyTest

import pytest

@pytest.mark.parametrize('x', [1, 2])
@pytest.mark.parametrize('y', [3, 4, 5])
def test_sum(x, y):
    assert x + y > 0

A 6

B 5

C 3

D 2

💡 Hint

Multiply the number of values in each parametrize decorator.

There are 2 values for x and 3 for y. The total tests run is 2 * 3 = 6.

❓ assertion

advanced

Correct assertion for parametrized test coverage

Which assertion correctly verifies that a parametrized test ran all expected cases?

PyTest

import pytest

@pytest.mark.parametrize('val', [10, 20, 30])
def test_positive(val):
    assert val > 0

A assert pytest.main(['-v']).count('test_positive') == 3

B assert pytest.main(['-v']).count('test_positive') == 0

C assert pytest.main(['-v']).count('test_positive') == 1

D assert pytest.main(['-v']).count('test_positive') == 6

💡 Hint

Count how many times the test function name appears in verbose output.

Since the test is parametrized with 3 values, it runs 3 times. The count should be 3.

🔧 Debug

advanced

Identify the error in parametrized test coverage

Why does this parametrized test not increase coverage as expected?

PyTest

import pytest

@pytest.mark.parametrize('a', [1, 2])
def test_example(a):
    assert a != 0

@pytest.mark.parametrize('b', [3, 4])
def test_example(b):
    assert b != 0

A Both tests run with 4 total cases, coverage is correct.

B The second test_example overwrites the first, so only 2 tests run.

C Parametrize decorators must be combined on one function to multiply coverage.

D The test names must be different to run both parametrized tests.

💡 Hint

Check if function names are unique in the test file.

Both functions have the same name, so the second overwrites the first. Only 2 tests run, not 4.

❓ framework

expert

How does pytest parametrize multiply coverage internally?

Which mechanism does pytest use internally to multiply test coverage when using @pytest.mark.parametrize?

A It runs the test once and caches results for all inputs.

B It modifies the test function code to loop over inputs during execution.

C It generates multiple test function calls with different arguments at collection time.

D It duplicates the test file for each parameter set before running tests.

💡 Hint

Think about when pytest creates separate test cases for each parameter.

Pytest creates multiple test calls with different arguments during test collection, so each runs independently.

Practice

1. What is the main benefit of using @pytest.mark.parametrize in tests?

easy

A. It reduces the number of test files needed.

B. It makes tests run faster by skipping some cases.

C. It automatically fixes bugs in the code.

D. It runs the same test multiple times with different inputs to increase coverage.

Solution

1. Step 1: Understand parametrize purpose

   The @pytest.mark.parametrize decorator runs one test function multiple times with different input values.

2. Step 2: Effect on test coverage

   Running tests with varied inputs increases coverage by checking more cases without writing extra test functions.

3. Final Answer:

   It runs the same test multiple times with different inputs to increase coverage. -> Option D

4. Quick Check:

   Parametrize = multiple inputs, more coverage [OK]

Hint: Parametrize repeats tests with different data for more coverage [OK]

Common Mistakes:

• Thinking parametrize skips tests
• Believing it fixes bugs automatically
• Confusing it with reducing test files

2. Which of the following is the correct syntax to parametrize a test with two inputs a and b in pytest?

easy

A. @pytest.mark.parametrize(a, b, [(1, 2), (3, 4)])

B. @pytest.mark.parametrize('a,b', [(1, 2), (3, 4)])

C. @pytest.mark.param('a,b', (1, 2), (3, 4))

D. @pytest.parametrize('a,b', [1, 2, 3, 4])

Solution

1. Step 1: Recall correct decorator name and parameters

   The correct decorator is @pytest.mark.parametrize with a string of parameter names and a list of tuples for values.

2. Step 2: Match syntax to options

   @pytest.mark.parametrize('a,b', [(1, 2), (3, 4)]) correctly uses @pytest.mark.parametrize('a,b', [(1, 2), (3, 4)]). Others have wrong names or formats.

3. Final Answer:

   @pytest.mark.parametrize('a,b', [(1, 2), (3, 4)]) -> Option B

4. Quick Check:

   Correct decorator and tuple list = @pytest.mark.parametrize('a,b', [(1, 2), (3, 4)]) [OK]

Hint: Use @pytest.mark.parametrize with string and list of tuples [OK]

Common Mistakes:

• Misspelling 'parametrize' as 'parametrize' or 'param'
• Passing parameters as separate arguments instead of a string
• Using list instead of list of tuples for multiple inputs

3. Given the following test code, how many times will the test run and what will be the output?

import pytest

@pytest.mark.parametrize('x', [0, 1, 2])
@pytest.mark.parametrize('y', [10, 20])
def test_sum(x, y):
    assert x + y > 10

medium

A. 6 runs; some fail because 0+10 is not > 10

B. 3 runs; all pass because only x is parametrize

C. 6 runs; all pass because sums are always > 10

D. 2 runs; all fail because y is less than 10

Solution

1. Step 1: Calculate total test runs

   Two parametrize decorators multiply: 3 values for x times 2 values for y = 6 runs total.

2. Step 2: Check assertion for each pair

   Pairs: (0,10), (0,20), (1,10), (1,20), (2,10), (2,20). Sum > 10 fails for 0+10=10 (not > 10), passes others.

3. Final Answer:

   6 runs; some fail because 0+10 is not > 10 -> Option A

4. Quick Check:

   Parametrize multiplies runs; check sums > 10 [OK]

Hint: Multiply parametrize counts; check each input pair assertion [OK]

Common Mistakes:

• Assuming runs equal to one parametrize only
• Ignoring assertion failure on boundary values
• Confusing > 10 with >= 10

4. Identify the error in this pytest parametrize usage:

@pytest.mark.parametrize('num', [1, 2, 3])
def test_is_even(num):
    assert num % 2 == 0

medium

A. The test will fail for odd numbers, but syntax is correct.

B. The parametrize decorator is missing parentheses.

C. The parameter name should be a list, not a string.

D. The assert statement syntax is invalid.

Solution

1. Step 1: Check syntax of parametrize

   The decorator syntax is correct: string parameter name and list of values.

2. Step 2: Analyze test logic

   The test asserts number is even. Since 1 and 3 are odd, test will fail for those inputs, but no syntax error.

3. Final Answer:

   The test will fail for odd numbers, but syntax is correct. -> Option A

4. Quick Check:

   Parametrize syntax correct; test logic causes failures [OK]

Hint: Syntax is fine; test logic causes failures on odd numbers [OK]

Common Mistakes:

• Thinking parametrize syntax is wrong
• Confusing parameter name type
• Misreading assert statement as syntax error

5. How does using multiple @pytest.mark.parametrize decorators on a single test function affect test coverage and execution?

hard

A. It causes syntax errors because multiple parametrize decorators are not allowed.

B. It runs tests only for matching pairs of parameters, reducing coverage.

C. It creates a Cartesian product of all parameter values, increasing coverage exponentially.

D. It runs tests sequentially for each decorator, but coverage stays the same.

Solution

1. Step 1: Understand multiple parametrize decorators

   Each decorator adds a set of parameter values. When stacked, pytest runs tests for every combination (Cartesian product) of these values.

2. Step 2: Effect on coverage and execution

   This multiplies test cases, increasing coverage exponentially and testing more input combinations.

3. Final Answer:

   It creates a Cartesian product of all parameter values, increasing coverage exponentially. -> Option C

4. Quick Check:

   Multiple parametrize = Cartesian product = more coverage [OK]

Hint: Stacked parametrize multiplies inputs as Cartesian product [OK]

Common Mistakes:

• Believing multiple decorators reduce coverage
• Thinking multiple decorators cause syntax errors
• Assuming tests run separately without combining inputs