Bird
Raised Fist0
Testing Fundamentalstesting~15 mins

Continuous Delivery testing in Testing Fundamentals - Build an Automation Script

Choose your learning style10 modes available
Verify automated deployment and testing in Continuous Delivery pipeline
Preconditions (4)
Step 1: Commit and push a code change to the main branch
Step 2: Wait for the CI pipeline to trigger automatically
Step 3: Verify that the build completes successfully
Step 4: Verify that automated tests run and pass in the CI pipeline
Step 5: Verify that the CD pipeline deploys the new build to the staging environment automatically
Step 6: Access the staging environment URL
Step 7: Verify that the deployed application is running and the new changes are visible
Step 8: Run a smoke test on the staging environment to confirm basic functionality
✅ Expected Result: The code change triggers the CI pipeline which builds and tests the code successfully. The CD pipeline automatically deploys the build to staging. The staging environment shows the updated application running correctly and passes the smoke test.
Automation Requirements - Python with Selenium and requests
Assertions Needed:
Build success status is confirmed from CI API
Test results indicate all tests passed
Deployment success status is confirmed from CD API
Staging environment URL is reachable
Application page contains expected updated content
Smoke test actions complete without errors
Best Practices:
Use API calls to verify CI/CD pipeline statuses instead of UI only
Use explicit waits for environment readiness
Separate test logic for build verification, deployment verification, and UI smoke test
Use Page Object Model for UI interactions
Handle network errors gracefully
Automated Solution
Testing Fundamentals
import requests
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

class StagingPage:
    def __init__(self, driver):
        self.driver = driver
    def open(self, url):
        self.driver.get(url)
    def is_updated_content_visible(self, expected_text):
        try:
            element = WebDriverWait(self.driver, 10).until(
                EC.visibility_of_element_located((By.XPATH, f"//*[contains(text(), '{expected_text}')]"))
            )
            return True
        except:
            return False

def verify_ci_build(ci_api_url, build_id, headers):
    response = requests.get(f"{ci_api_url}/builds/{build_id}", headers=headers)
    response.raise_for_status()
    data = response.json()
    return data.get('status') == 'success'

def verify_ci_tests(ci_api_url, build_id, headers):
    response = requests.get(f"{ci_api_url}/builds/{build_id}/tests", headers=headers)
    response.raise_for_status()
    data = response.json()
    return all(test['status'] == 'passed' for test in data.get('tests', []))

def verify_cd_deployment(cd_api_url, deployment_id, headers):
    response = requests.get(f"{cd_api_url}/deployments/{deployment_id}", headers=headers)
    response.raise_for_status()
    data = response.json()
    return data.get('status') == 'deployed'

def smoke_test_staging(staging_url):
    driver = webdriver.Chrome()
    page = StagingPage(driver)
    try:
        page.open(staging_url)
        if not page.is_updated_content_visible('New Feature'):
            return False
        # Additional smoke test steps can be added here
        return True
    finally:
        driver.quit()

# Example usage
ci_api_url = 'https://ci.example.com/api'
cd_api_url = 'https://cd.example.com/api'
headers = {'Authorization': 'Bearer your_token_here'}
build_id = '12345'
deployment_id = '67890'
staging_url = 'https://staging.example.com'

assert verify_ci_build(ci_api_url, build_id, headers), 'CI build failed'
assert verify_ci_tests(ci_api_url, build_id, headers), 'CI tests failed'
assert verify_cd_deployment(cd_api_url, deployment_id, headers), 'CD deployment failed'
assert smoke_test_staging(staging_url), 'Smoke test failed on staging environment'

This script automates the verification of a Continuous Delivery pipeline.

First, it uses requests to call the CI API and check if the build succeeded and all tests passed. Then it calls the CD API to confirm the deployment status.

For the staging environment, it uses Selenium WebDriver to open the application URL and check if the updated content is visible, simulating a smoke test.

The code uses explicit waits to ensure elements are loaded before checking. It separates concerns by having functions for each verification step and a Page Object for UI interaction.

Assertions ensure the test fails clearly if any step does not meet expectations.

Common Mistakes - 4 Pitfalls
Checking deployment success only by UI without verifying CI/CD pipeline status via API
Using fixed sleep delays instead of explicit waits for page elements
Combining all test steps in one large function without separation
Not handling network errors or API failures gracefully
Bonus Challenge

Now add data-driven testing with 3 different build and deployment IDs to verify multiple pipeline runs

Show Hint

Practice

(1/5)
1. What is the main goal of Continuous Delivery testing in software development?
easy
A. To delay testing until after deployment
B. To write manual test cases for every feature
C. To ensure the software is always in a releasable state
D. To test only the user interface manually

Solution

  1. Step 1: Understand Continuous Delivery testing purpose

    Continuous Delivery testing aims to keep software ready to release at any time by running automated tests frequently.
  2. Step 2: Compare options with the goal

    Only To ensure the software is always in a releasable state matches this goal. Other options describe manual or delayed testing, which contradict Continuous Delivery principles.
  3. Final Answer:

    To ensure the software is always in a releasable state -> Option C
  4. Quick Check:

    Continuous Delivery = Always releasable software [OK]
Hint: Continuous Delivery means software is always ready to release [OK]
Common Mistakes:
  • Thinking manual testing is the main focus
  • Believing testing happens only after deployment
  • Confusing Continuous Delivery with manual UI testing
2. Which of the following is the correct syntax to define a simple automated test step in a Continuous Delivery pipeline script?
easy
A. run_tests script pytest tests/
B. run_tests: script: pytest tests/
C. run_tests: pytest tests/
D. run_tests = pytest tests/

Solution

  1. Step 1: Identify correct YAML syntax for pipeline steps

    Pipeline scripts often use YAML format where keys have colons and nested keys are indented, e.g., 'run_tests: script: pytest tests/'.
  2. Step 2: Check each option's syntax

    run_tests: script: pytest tests/ uses correct YAML with colon and nested script key. Options A, C, and D miss colons or use invalid assignment.
  3. Final Answer:

    run_tests: script: pytest tests/ -> Option B
  4. Quick Check:

    YAML step syntax = run_tests: script: command [OK]
Hint: YAML keys need colons and proper indentation [OK]
Common Mistakes:
  • Omitting colons in YAML keys
  • Using equal signs instead of colons
  • Not indenting nested keys properly
3. Consider this simplified test pipeline code snippet:
tests = ['unit', 'integration', 'e2e']
results = []
for test in tests:
    if test == 'integration':
        results.append('fail')
    else:
        results.append('pass')
print(results)

What will be the output after running this code?
medium
A. ['pass', 'fail', 'pass']
B. ['fail', 'fail', 'fail']
C. ['pass', 'pass', 'fail']
D. ['fail', 'pass', 'fail']

Solution

  1. Step 1: Analyze the loop and conditions

    The list 'tests' has three items: 'unit', 'integration', 'e2e'. The loop checks if the test is 'integration' to append 'fail', else 'pass'.
  2. Step 2: Determine results for each test

    'unit' -> not 'integration' -> 'pass'
    'integration' -> equals 'integration' -> 'fail'
    'e2e' -> not 'integration' -> 'pass'
  3. Final Answer:

    ['pass', 'fail', 'pass'] -> Option A
  4. Quick Check:

    Only 'integration' test fails [OK]
Hint: Only 'integration' test appends 'fail' [OK]
Common Mistakes:
  • Assuming all tests pass
  • Confusing the condition logic
  • Mixing order of results
4. A Continuous Delivery pipeline script has this step:
test_step:
  script:
    - pytest --maxfail=1 --disable-warnings
    - echo "Tests completed"

However, the pipeline does not stop on the first test failure as expected. What is the likely cause?
medium
A. pytest does not support the --maxfail option
B. The --maxfail=1 option is invalid and ignored
C. The echo command causes the pipeline to stop early
D. The pipeline runs all script commands regardless of failure by default

Solution

  1. Step 1: Understand pipeline script behavior

    In many CI/CD tools, all commands under 'script' run sequentially even if one fails, unless configured to stop on failure.
  2. Step 2: Analyze the options and commands

    --maxfail=1 is a valid pytest option to stop pytest early, but the pipeline still runs the next command 'echo'. So the pipeline itself does not stop on failure automatically.
  3. Final Answer:

    The pipeline runs all script commands regardless of failure by default -> Option D
  4. Quick Check:

    Pipeline runs all script lines unless configured to stop [OK]
Hint: Pipeline runs all script commands unless told to stop [OK]
Common Mistakes:
  • Thinking pytest controls pipeline flow
  • Believing echo causes failure
  • Assuming invalid option causes issue
5. In a Continuous Delivery pipeline, you want to run tests only on changed files to speed up feedback. Which approach best achieves this?
hard
A. Use a script to detect changed files and run tests related to those files
B. Run all tests every time regardless of changes
C. Manually select tests to run after each commit
D. Disable automated tests and rely on manual testing

Solution

  1. Step 1: Understand test optimization in Continuous Delivery

    Running all tests every time can be slow. Detecting changed files and running only related tests speeds feedback.
  2. Step 2: Evaluate options for automation and speed

    Use a script to detect changed files and run tests related to those files automates detection and selective testing, fitting Continuous Delivery goals. Options B, C, and D are inefficient or manual.
  3. Final Answer:

    Use a script to detect changed files and run tests related to those files -> Option A
  4. Quick Check:

    Selective testing on changes = faster feedback [OK]
Hint: Run tests only for changed files to save time [OK]
Common Mistakes:
  • Running all tests every time without optimization
  • Relying on manual test selection
  • Skipping automated tests entirely
Recommended

Ready to prove it?

Pass the test and this pattern moves unknown strong while earning +1 Grip toward your certificate.

10 questions~5 mineasy hard