Continuous Delivery testing in Testing Fundamentals - Test Execution Trace

Test Overview

This test simulates a Continuous Delivery pipeline step where the application is automatically deployed to a staging environment and verified for successful deployment. It checks that the deployment page loads and shows a success message, ensuring the delivery process works smoothly.

Test Code - Selenium with unittest

Testing Fundamentals

import unittest
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class TestContinuousDeliveryDeployment(unittest.TestCase):
    def setUp(self):
        self.driver = webdriver.Chrome()
        self.driver.implicitly_wait(5)

    def test_deployment_success_message(self):
        driver = self.driver
        driver.get('https://staging.example.com/deployment-status')

        # Wait until the deployment status element is visible
        status_element = WebDriverWait(driver, 10).until(
            EC.visibility_of_element_located((By.ID, 'deployment-status'))
        )

        # Verify the deployment success message is displayed
        self.assertIn('Deployment Successful', status_element.text)

    def tearDown(self):
        self.driver.quit()

if __name__ == '__main__':
    unittest.main()

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Test starts and Chrome browser opens	Chrome browser window is open and ready	—	PASS
2	Browser navigates to 'https://staging.example.com/deployment-status'	Deployment status page loads in the browser	—	PASS
3	WebDriverWait waits up to 10 seconds for element with ID 'deployment-status' to be visible	Deployment status element is visible on the page	Check element with ID 'deployment-status' is present and visible	PASS
4	Assert that the text 'Deployment Successful' is present in the deployment status element	Deployment status element contains the success message text	Verify 'Deployment Successful' text is in element text	PASS
5	Browser closes and test ends	Browser window is closed	—	PASS

Failure Scenario

Failing Condition: The deployment status element does not appear or does not contain the success message

Execution Trace Quiz - 3 Questions

What does the test verify on the deployment status page?

A That the login form is present

B That the page shows 'Deployment Successful' message

C That the browser can open any website

D That the deployment button is clickable

Key Result

Always wait explicitly for key elements in Continuous Delivery tests to ensure the deployment page is fully loaded before checking success messages. This prevents flaky tests caused by timing issues.

Practice

1. What is the main goal of Continuous Delivery testing in software development?

easy

A. To delay testing until after deployment

B. To write manual test cases for every feature

C. To ensure the software is always in a releasable state

D. To test only the user interface manually

Solution

1. Step 1: Understand Continuous Delivery testing purpose

   Continuous Delivery testing aims to keep software ready to release at any time by running automated tests frequently.

2. Step 2: Compare options with the goal

   Only To ensure the software is always in a releasable state matches this goal. Other options describe manual or delayed testing, which contradict Continuous Delivery principles.

3. Final Answer:

   To ensure the software is always in a releasable state -> Option C

4. Quick Check:

   Continuous Delivery = Always releasable software [OK]

Hint: Continuous Delivery means software is always ready to release [OK]

Common Mistakes:

• Thinking manual testing is the main focus
• Believing testing happens only after deployment
• Confusing Continuous Delivery with manual UI testing

2. Which of the following is the correct syntax to define a simple automated test step in a Continuous Delivery pipeline script?

easy

A. run_tests script pytest tests/

B. run_tests: script: pytest tests/

C. run_tests: pytest tests/

D. run_tests = pytest tests/

Solution

1. Step 1: Identify correct YAML syntax for pipeline steps

   Pipeline scripts often use YAML format where keys have colons and nested keys are indented, e.g., 'run_tests: script: pytest tests/'.

2. Step 2: Check each option's syntax

   run_tests: script: pytest tests/ uses correct YAML with colon and nested script key. Options A, C, and D miss colons or use invalid assignment.

3. Final Answer:

   run_tests: script: pytest tests/ -> Option B

4. Quick Check:

   YAML step syntax = run_tests: script: command [OK]

Hint: YAML keys need colons and proper indentation [OK]

Common Mistakes:

• Omitting colons in YAML keys
• Using equal signs instead of colons
• Not indenting nested keys properly

3. Consider this simplified test pipeline code snippet:

tests = ['unit', 'integration', 'e2e']
results = []
for test in tests:
    if test == 'integration':
        results.append('fail')
    else:
        results.append('pass')
print(results)

What will be the output after running this code?

medium

A. ['pass', 'fail', 'pass']

B. ['fail', 'fail', 'fail']

C. ['pass', 'pass', 'fail']

D. ['fail', 'pass', 'fail']

Solution

1. Step 1: Analyze the loop and conditions

   The list 'tests' has three items: 'unit', 'integration', 'e2e'. The loop checks if the test is 'integration' to append 'fail', else 'pass'.

2. Step 2: Determine results for each test

   'unit' -> not 'integration' -> 'pass'
   'integration' -> equals 'integration' -> 'fail'
   'e2e' -> not 'integration' -> 'pass'

3. Final Answer:

   ['pass', 'fail', 'pass'] -> Option A

4. Quick Check:

   Only 'integration' test fails [OK]

Hint: Only 'integration' test appends 'fail' [OK]

Common Mistakes:

• Assuming all tests pass
• Confusing the condition logic
• Mixing order of results

4. A Continuous Delivery pipeline script has this step:

test_step:
  script:
    - pytest --maxfail=1 --disable-warnings
    - echo "Tests completed"

However, the pipeline does not stop on the first test failure as expected. What is the likely cause?

medium

A. pytest does not support the --maxfail option

B. The --maxfail=1 option is invalid and ignored

C. The echo command causes the pipeline to stop early

D. The pipeline runs all script commands regardless of failure by default

Solution

1. Step 1: Understand pipeline script behavior

   In many CI/CD tools, all commands under 'script' run sequentially even if one fails, unless configured to stop on failure.

2. Step 2: Analyze the options and commands

   --maxfail=1 is a valid pytest option to stop pytest early, but the pipeline still runs the next command 'echo'. So the pipeline itself does not stop on failure automatically.

3. Final Answer:

   The pipeline runs all script commands regardless of failure by default -> Option D

4. Quick Check:

   Pipeline runs all script lines unless configured to stop [OK]

Hint: Pipeline runs all script commands unless told to stop [OK]

Common Mistakes:

• Thinking pytest controls pipeline flow
• Believing echo causes failure
• Assuming invalid option causes issue

5. In a Continuous Delivery pipeline, you want to run tests only on changed files to speed up feedback. Which approach best achieves this?

hard

A. Use a script to detect changed files and run tests related to those files

B. Run all tests every time regardless of changes

C. Manually select tests to run after each commit

D. Disable automated tests and rely on manual testing

Solution

1. Step 1: Understand test optimization in Continuous Delivery

   Running all tests every time can be slow. Detecting changed files and running only related tests speeds feedback.

2. Step 2: Evaluate options for automation and speed

   Use a script to detect changed files and run tests related to those files automates detection and selective testing, fitting Continuous Delivery goals. Options B, C, and D are inefficient or manual.

3. Final Answer:

   Use a script to detect changed files and run tests related to those files -> Option A

4. Quick Check:

   Selective testing on changes = faster feedback [OK]

Hint: Run tests only for changed files to save time [OK]

Common Mistakes:

• Running all tests every time without optimization
• Relying on manual test selection
• Skipping automated tests entirely