Why documentation ensures repeatability in Testing Fundamentals - Automation Benefits in Action

Verify that a test case can be repeated exactly using documentation

Preconditions (2)

Step 1: Read the test case documentation carefully

Step 2: Set up the test environment exactly as described

Step 3: Follow each step in the documentation to execute the test

Step 4: Record the actual results

Step 5: Repeat the test by following the documentation again without changes

Step 6: Compare the results of both test runs

✅ Expected Result: Both test runs produce the same results, showing that documentation ensures repeatability

Automation Requirements - pytest

Assertions Needed:

Verify that the test steps are executed in the correct order

Verify that the test environment matches the documented setup

Verify that the test results are consistent across multiple runs

Best Practices:

Use clear and descriptive test step comments

Use setup and teardown methods to prepare and clean the environment

Use assertions to validate expected outcomes

Keep test code simple and readable

Automated Solution

Testing Fundamentals

import pytest

class TestDocumentationRepeatability:
    def setup_method(self):
        # Setup environment as per documentation
        self.environment_ready = True
        self.test_data = [1, 2, 3]

    def teardown_method(self):
        # Clean up environment
        self.environment_ready = False
        self.test_data = []

    def test_repeatability(self):
        # Step 1: Verify environment setup
        assert self.environment_ready, "Environment is not set up correctly"

        # Step 2: Execute test steps as documented
        result_first_run = self.execute_test_steps(self.test_data)

        # Step 3: Repeat test steps
        result_second_run = self.execute_test_steps(self.test_data)

        # Step 4: Verify results are the same
        assert result_first_run == result_second_run, "Test results differ between runs"

    def execute_test_steps(self, data):
        # Simulate test steps from documentation
        processed = [x * 2 for x in data]  # Example operation
        return processed

This test class simulates following documented test steps exactly. The setup_method prepares the environment as described in the documentation. The test_repeatability method runs the test steps twice by calling execute_test_steps with the same data. Assertions check that the environment is ready and that results from both runs are identical, proving repeatability. The teardown_method cleans up after tests to keep environment consistent for next runs.

This approach shows how clear documentation allows anyone to repeat tests and get the same results, which is key for reliable testing.

Common Mistakes - 3 Pitfalls

Not setting up the environment exactly as documented

Skipping cleanup after tests

Not verifying results between runs

Bonus Challenge

Now add data-driven testing with 3 different input sets to verify repeatability across varied data

Show Hint

Practice

1. Why is documentation important for repeatability in software testing?

easy

A. It automatically fixes bugs found during testing

B. It makes the test run faster on all machines

C. It clearly describes test steps and expected results for others to follow

D. It reduces the need for testers to communicate

Solution

1. Step 1: Understand the role of documentation

   Documentation provides clear instructions and expected outcomes for tests.

2. Step 2: Connect documentation to repeatability

   Clear steps allow anyone to run the test the same way, ensuring repeatability.

3. Final Answer:

   It clearly describes test steps and expected results for others to follow -> Option C

4. Quick Check:

   Documentation = Clear steps = Repeatability [OK]

Hint: Documentation means clear steps and results for repeat tests [OK]

Common Mistakes:

• Thinking documentation speeds up tests
• Believing documentation fixes bugs automatically
• Assuming documentation replaces communication

2. Which of the following is the correct way to document a test step?

easy

A. Run the test and see what happens

B. Click the 'Submit' button and expect a confirmation message

C. Guess the expected result based on intuition

D. Skip writing steps if the test is simple

Solution

1. Step 1: Identify proper test step documentation

   Test steps should be clear actions with expected outcomes.

2. Step 2: Evaluate options for clarity and correctness

   Click the 'Submit' button and expect a confirmation message clearly states the action and expected result, making it correct.

3. Final Answer:

   Click the 'Submit' button and expect a confirmation message -> Option B

4. Quick Check:

   Clear action + expected result = Correct documentation [OK]

Hint: Document clear actions and expected results [OK]

Common Mistakes:

• Writing vague or incomplete steps
• Skipping expected results
• Assuming testers will guess outcomes

3. Consider this test documentation snippet:

1. Open login page
2. Enter username 'user1'
3. Enter password 'pass123'
4. Click 'Login'
Expected result: User dashboard appears

What is the main benefit of this documentation?

medium

A. It allows anyone to repeat the test exactly and verify the same result

B. It guarantees the test will find all bugs in the login feature

C. It makes the test run faster on different browsers

D. It removes the need for testers to check the dashboard manually

Solution

1. Step 1: Analyze the test steps and expected result

   The steps are clear and detailed, describing exactly what to do and expect.

2. Step 2: Understand the benefit of clear documentation

   Clear steps let anyone run the test the same way and check if the dashboard appears.

3. Final Answer:

   It allows anyone to repeat the test exactly and verify the same result -> Option A

4. Quick Check:

   Clear steps + expected result = Repeatability [OK]

Hint: Clear steps let anyone repeat and verify results [OK]

Common Mistakes:

• Assuming documentation finds all bugs
• Thinking documentation speeds up tests
• Believing manual checks are unnecessary

4. A tester wrote this documentation:

1. Click 'Submit'
2. Expect success message

What is the main problem here?

medium

A. The test will run too slowly

B. The documentation is too long and complicated

C. The expected result is incorrect

D. The steps are too vague and lack details for repeatability

Solution

1. Step 1: Review the test steps for clarity

   The steps do not specify which 'Submit' button or what success message exactly.

2. Step 2: Understand impact on repeatability

   Vague steps make it hard for others to repeat the test reliably.

3. Final Answer:

   The steps are too vague and lack details for repeatability -> Option D

4. Quick Check:

   Vague steps = Poor repeatability [OK]

Hint: Vague steps cause repeatability problems [OK]

Common Mistakes:

• Thinking test speed is affected by documentation
• Assuming expected result is always wrong if vague
• Believing longer docs are always bad

5. You have a test documented as:

1. Navigate to user profile
2. Update email to '[e-mail]'
3. Save changes
Expected: Confirmation message appears

Later, the app changes to require email verification after update. How should you update the documentation to keep repeatability?

hard

A. Add a step to check the verification email and confirm it

B. Remove the expected confirmation message

C. Keep the documentation as is because the test passed before

D. Skip the test since the app changed

Solution

1. Step 1: Identify the app change impact

   The app now requires email verification after update, changing expected behavior.

2. Step 2: Update documentation to reflect new steps

   Add steps to check and confirm the verification email to keep the test accurate and repeatable.

3. Final Answer:

   Add a step to check the verification email and confirm it -> Option A

4. Quick Check:

   Update docs with new steps = Maintain repeatability [OK]

Hint: Update docs when app changes to keep tests repeatable [OK]

Common Mistakes:

• Ignoring app changes in documentation
• Removing expected results without replacement
• Skipping tests instead of updating docs