Deterministic tests in PyTest - Cheat Sheet & Quick Revision

Recall & Review

beginner

What does it mean for a test to be deterministic?

A deterministic test always produces the same result when run with the same code and inputs, no matter how many times it runs.

beginner

Why are deterministic tests important in software testing?

They help ensure reliability and trust in test results because failures are consistent and reproducible, making debugging easier.

intermediate

How can randomness affect test determinism?

Randomness can cause tests to behave differently each run, leading to flaky tests that sometimes pass and sometimes fail.

intermediate

What is a common way to make tests deterministic when they use random values?

Set a fixed seed for the random number generator so it produces the same sequence of values every time the test runs.

beginner

Example: In pytest, how do you fix randomness to make a test deterministic?

Use code like <code>import random
random.seed(42)</code> at the start of your test to fix the random seed.

What is a key characteristic of a deterministic test?

A It sometimes passes and sometimes fails unpredictably.

B It uses random values to test different cases.

C It always produces the same result given the same inputs.

D It runs only once and never again.

Deterministic tests always produce the same result with the same inputs, ensuring reliability.

Which practice helps make tests deterministic when randomness is involved?

A Ignoring random values in tests.

B Using different random seeds each run.

C Running tests multiple times to average results.

D Setting a fixed seed for the random number generator.

Setting a fixed seed ensures the random values are the same every run, making tests deterministic.

What problem do non-deterministic (flaky) tests cause?

A They make debugging harder because failures are inconsistent.

B They always pass, hiding bugs.

C They run faster than deterministic tests.

D They require no maintenance.

Flaky tests fail unpredictably, making it hard to know if a failure is real or random.

In pytest, where should you set the random seed to ensure determinism?

A Only in the main application code.

B Inside the test function before using random values.

C After the test finishes.

D You cannot set a random seed in pytest.

Setting the seed inside the test ensures the random values are predictable for that test run.

Which of these is NOT a cause of non-deterministic tests?

A Using fixed inputs and no randomness.

B Relying on external systems with changing data.

C Using random values without fixed seeds.

D Tests depending on time or date.

Using fixed inputs and no randomness helps keep tests deterministic.

Explain what deterministic tests are and why they matter in software testing.

Think about how tests behave when run multiple times with the same code.

Describe how you would make a pytest test deterministic if it uses random numbers.

Consider the random.seed() function and its placement.