Pipeline stages and test gates in Testing Fundamentals - Build an Automation Script

Verify pipeline stages and test gates enforce quality checks

Preconditions (2)

Step 1: Trigger a pipeline run with a code change

Step 2: Observe the Build stage completes successfully

Step 3: Verify Unit Test stage runs and passes all tests

Step 4: If Unit Test passes, verify Integration Test stage runs

Step 5: If Integration Test passes, verify Deploy stage runs

Step 6: If any test stage fails, verify pipeline stops and does not proceed to next stage

✅ Expected Result: Pipeline stages execute in order with test gates blocking progression on failures. Deployment only occurs if all tests pass.

Automation Requirements - Python with pytest and requests

Assertions Needed:

Build stage status is success

Unit Test stage status is success

Integration Test stage status is success

Deploy stage runs only if all previous stages succeed

Pipeline stops immediately on any test failure

Best Practices:

Use API calls to query pipeline stage statuses

Implement explicit checks for each stage result

Fail test immediately on any stage failure

Use clear and descriptive assertion messages

Automated Solution

Testing Fundamentals

import requests
import pytest

PIPELINE_API_URL = "https://ci.example.com/api/pipelines/12345/status"

@pytest.fixture
async def get_pipeline_status():
    response = requests.get(PIPELINE_API_URL)
    response.raise_for_status()
    return response.json()

@pytest.mark.asyncio
async def test_pipeline_stages_and_gates(get_pipeline_status):
    status = get_pipeline_status
    # Check Build stage
    build_stage = next((stage for stage in status['stages'] if stage['name'] == 'Build'), None)
    assert build_stage is not None, "Build stage not found"
    assert build_stage['status'] == 'success', f"Build stage failed with status {build_stage['status']}"

    # Check Unit Test stage
    unit_test_stage = next((stage for stage in status['stages'] if stage['name'] == 'Unit Test'), None)
    assert unit_test_stage is not None, "Unit Test stage not found"
    assert unit_test_stage['status'] == 'success', f"Unit Test stage failed with status {unit_test_stage['status']}"

    # Check Integration Test stage
    integration_test_stage = next((stage for stage in status['stages'] if stage['name'] == 'Integration Test'), None)
    assert integration_test_stage is not None, "Integration Test stage not found"
    assert integration_test_stage['status'] == 'success', f"Integration Test stage failed with status {integration_test_stage['status']}"

    # Check Deploy stage only runs if all previous stages succeeded
    deploy_stage = next((stage for stage in status['stages'] if stage['name'] == 'Deploy'), None)
    assert deploy_stage is not None, "Deploy stage not found"
    assert deploy_stage['status'] == 'running' or deploy_stage['status'] == 'success', f"Deploy stage did not run properly, status: {deploy_stage['status']}"

    # Verify pipeline stops on failure (simulate by checking no stages after failure)
    failed_stages = [stage for stage in status['stages'] if stage['status'] == 'failed']
    if failed_stages:
        failed_index = status['stages'].index(failed_stages[0])
        stages_after_failure = status['stages'][failed_index + 1:]
        assert all(stage['status'] == 'not_run' for stage in stages_after_failure), "Pipeline did not stop after failure"

This test script uses Python with pytest and requests to automate checking a CI/CD pipeline's stages and test gates.

First, it calls the pipeline API to get the current status of all stages.

It then verifies each stage exists and has the expected status. The Build, Unit Test, and Integration Test stages must all succeed.

The Deploy stage should only run if all previous stages succeeded, so its status is checked to be either running or success.

If any stage failed, the test confirms that no subsequent stages ran, ensuring the pipeline stops correctly at the test gate.

Assertions have clear messages to help understand failures.

This approach uses explicit checks and API queries, following best practices for test automation of pipelines.

Common Mistakes - 4 Pitfalls

{'mistake': 'Hardcoding stage statuses without querying live pipeline data', 'why_bad': "The test won't reflect the real pipeline state and may pass incorrectly.", 'correct_approach': 'Use API calls to get current pipeline stage statuses dynamically.'}

Not checking if stages exist before asserting status

Ignoring pipeline stop behavior on failure

Using sleep or fixed waits instead of checking stage status

Bonus Challenge

Now add data-driven testing with 3 different pipeline IDs to verify multiple pipelines

Show Hint

Practice

1. What is the main purpose of test gates in a software delivery pipeline?

easy

A. To write new test cases for the software

B. To check if tests pass before moving to the next stage

C. To deploy the software to production automatically

D. To monitor user feedback after release

Solution

1. Step 1: Understand the role of test gates

   Test gates act as checkpoints that verify if tests have passed successfully before allowing the pipeline to continue.

2. Step 2: Differentiate from other pipeline actions

   Deploying software or writing tests are separate activities, not the main function of test gates.

3. Final Answer:

   To check if tests pass before moving to the next stage -> Option B

4. Quick Check:

   Test gates = verify tests pass [OK]

Hint: Test gates stop pipeline if tests fail [OK]

Common Mistakes:

• Confusing test gates with deployment steps
• Thinking test gates create tests
• Assuming test gates handle user feedback

2. Which of the following is the correct order of pipeline stages in a typical CI/CD pipeline?

easy

A. Test -> Build -> Deploy -> Release

B. Release -> Build -> Test -> Deploy

C. Deploy -> Build -> Test -> Release

D. Build -> Test -> Deploy -> Release

Solution

1. Step 1: Recall standard pipeline flow

   The usual order is to first build the software, then run tests, deploy to an environment, and finally release to users.

2. Step 2: Match options to this order

   Build -> Test -> Deploy -> Release matches this sequence exactly.

3. Final Answer:

   Build -> Test -> Deploy -> Release -> Option D

4. Quick Check:

   Build before Test before Deploy [OK]

Hint: Build comes before testing in pipelines [OK]

Common Mistakes:

• Swapping build and test stages
• Deploying before testing
• Releasing before deployment

3. Consider this simplified pipeline code snippet:

stages:
  - build
  - test
  - deploy

test_gate:
  stage: test
  condition: tests_passed == true
  action: continue_pipeline

If tests_passed is false, what happens next?

medium

A. Pipeline stops and does not proceed to deploy

B. Pipeline skips test stage and goes to deploy

C. Pipeline continues to deploy regardless

D. Pipeline restarts from build stage

Solution

1. Step 1: Analyze the test gate condition

   The test gate checks if tests_passed == true. If false, the condition fails.

2. Step 2: Understand pipeline behavior on failed test gate

   When the condition fails, the pipeline stops and does not continue to the deploy stage.

3. Final Answer:

   Pipeline stops and does not proceed to deploy -> Option A

4. Quick Check:

   Failed test gate = pipeline stops [OK]

Hint: Failed test gate stops pipeline flow [OK]

Common Mistakes:

• Assuming pipeline skips tests
• Thinking pipeline restarts automatically
• Believing deploy runs despite failed tests

4. You have this pipeline snippet:

stages:
  - build
  - test
  - deploy

test_gate:
  stage: test
  condition: tests_passed
  action: continue_pipeline

But the pipeline always proceeds even when tests fail. What is the likely error?

medium

A. The condition should explicitly check for true, e.g., tests_passed == true

B. The deploy stage is missing

C. The build stage is not defined

D. The test gate action should be 'stop_pipeline' instead of 'continue_pipeline'

Solution

1. Step 1: Examine the test gate condition

   The condition uses tests_passed alone, which may not evaluate correctly if it's not a boolean.

2. Step 2: Correct the condition syntax

   Explicitly checking tests_passed == true ensures the gate only passes when tests truly pass.

3. Final Answer:

   The condition should explicitly check for true, e.g., tests_passed == true -> Option A

4. Quick Check:

   Explicit condition check fixes gate logic [OK]

Hint: Use explicit boolean checks in test gates [OK]

Common Mistakes:

• Ignoring condition syntax errors
• Assuming missing stages cause gate failure
• Confusing action names in test gates

5. A pipeline has these stages:

1. Build
2. Unit Test
3. Integration Test
4. Deploy

Test gates are set after Unit Test and Integration Test stages. If the Integration Test gate fails, what should happen to ensure safe deployment?

hard

A. Deploy only if Unit Test gate passed, ignoring Integration Test

B. Skip Integration Test and deploy anyway

C. Stop the pipeline and prevent deployment

D. Restart the pipeline from Build stage automatically

Solution

1. Step 1: Understand test gate purpose after Integration Test

   The gate ensures that integration tests pass before deployment to avoid faulty releases.

2. Step 2: Determine correct pipeline behavior on gate failure

   If the Integration Test gate fails, the pipeline must stop to prevent deploying broken software.

3. Final Answer:

   Stop the pipeline and prevent deployment -> Option C

4. Quick Check:

   Failing test gate = stop deployment [OK]

Hint: Failing integration tests block deployment [OK]

Common Mistakes:

• Ignoring integration test failures
• Assuming automatic pipeline restart
• Deploying despite failed test gates