Pipeline stages and test gates in Testing Fundamentals - Framework Patterns

Framework Mode - Pipeline stages and test gates

Pipeline Stages Folder Structure

  pipeline-project/
  ├── src/
  │   └── main/               # Application source code
  ├── tests/                  # Automated tests
  │   ├── unit/               # Unit tests
  │   ├── integration/        # Integration tests
  │   ├── e2e/                # End-to-end tests
  │   └── performance/        # Performance tests
  ├── ci/                     # CI/CD pipeline scripts and configs
  │   ├── Jenkinsfile         # Pipeline definition
  │   ├── test-gates/         # Scripts for test gates
  │   │   ├── unit-gate.sh
  │   │   ├── integration-gate.sh
  │   │   └── e2e-gate.sh
  │   └── deploy/             # Deployment scripts
  ├── reports/                # Test reports output
  └── config/                 # Environment and pipeline configs
      ├── dev.yaml
      ├── staging.yaml
      └── prod.yaml
  

Pipeline Stages and Test Gates Layers

• Source Code Layer: Application code under src/main.
• Test Layer: Automated tests organized by type (unit, integration, e2e, performance).
• Pipeline Layer: CI/CD scripts defining stages like build, test, deploy.
• Test Gate Layer: Scripts or jobs that act as quality checkpoints after each test stage.
• Reporting Layer: Collects and stores test results and coverage reports.
• Configuration Layer: Holds environment-specific settings and pipeline parameters.

Configuration Patterns for Pipeline and Test Gates

• Environment Files: Use separate YAML/JSON files for dev, staging, prod to manage URLs, credentials, and flags.
• Pipeline Variables: Define variables in pipeline scripts or CI tool to toggle stages or test gates.
• Test Gate Thresholds: Configure pass/fail criteria like minimum test coverage or zero critical failures.
• Secrets Management: Store sensitive data securely using CI/CD secrets or vaults, not in code.
• Parameterization: Allow pipeline stages to accept parameters for flexible execution (e.g., run only unit tests).

Test Reporting and CI/CD Integration

• Test Reports: Generate HTML or XML reports after each test stage for visibility.
• Gate Results: Test gate scripts analyze reports and decide pass/fail to allow pipeline continuation.
• Notifications: Send alerts (email, Slack) on gate failures to inform the team immediately.
• CI/CD Integration: Pipeline stages run tests and gates sequentially; failure stops deployment.
• Dashboard: Use CI tools dashboards to track pipeline status and test gate outcomes over time.

Best Practices for Pipeline Stages and Test Gates

1. Fail Fast: Design test gates to quickly stop the pipeline on critical failures to save resources.
2. Clear Criteria: Define explicit pass/fail rules for each test gate to avoid confusion.
3. Incremental Testing: Run fast unit tests early, then slower integration and e2e tests later.
4. Automate Everything: Automate test execution, gate evaluation, and reporting for consistency.
5. Visibility: Provide clear, accessible reports and notifications for all stakeholders.

Self Check Question

In the pipeline-project folder structure, where would you add a new test gate script for API contract testing?

Key Result

Organize pipeline stages with clear test gates to ensure quality checkpoints before deployment.

Practice

1. What is the main purpose of test gates in a software delivery pipeline?

easy

A. To write new test cases for the software

B. To check if tests pass before moving to the next stage

C. To deploy the software to production automatically

D. To monitor user feedback after release

Solution

1. Step 1: Understand the role of test gates

   Test gates act as checkpoints that verify if tests have passed successfully before allowing the pipeline to continue.

2. Step 2: Differentiate from other pipeline actions

   Deploying software or writing tests are separate activities, not the main function of test gates.

3. Final Answer:

   To check if tests pass before moving to the next stage -> Option B

4. Quick Check:

   Test gates = verify tests pass [OK]

Hint: Test gates stop pipeline if tests fail [OK]

Common Mistakes:

• Confusing test gates with deployment steps
• Thinking test gates create tests
• Assuming test gates handle user feedback

2. Which of the following is the correct order of pipeline stages in a typical CI/CD pipeline?

easy

A. Test -> Build -> Deploy -> Release

B. Release -> Build -> Test -> Deploy

C. Deploy -> Build -> Test -> Release

D. Build -> Test -> Deploy -> Release

Solution

1. Step 1: Recall standard pipeline flow

   The usual order is to first build the software, then run tests, deploy to an environment, and finally release to users.

2. Step 2: Match options to this order

   Build -> Test -> Deploy -> Release matches this sequence exactly.

3. Final Answer:

   Build -> Test -> Deploy -> Release -> Option D

4. Quick Check:

   Build before Test before Deploy [OK]

Hint: Build comes before testing in pipelines [OK]

Common Mistakes:

• Swapping build and test stages
• Deploying before testing
• Releasing before deployment

3. Consider this simplified pipeline code snippet:

stages:
  - build
  - test
  - deploy

test_gate:
  stage: test
  condition: tests_passed == true
  action: continue_pipeline

If tests_passed is false, what happens next?

medium

A. Pipeline stops and does not proceed to deploy

B. Pipeline skips test stage and goes to deploy

C. Pipeline continues to deploy regardless

D. Pipeline restarts from build stage

Solution

1. Step 1: Analyze the test gate condition

   The test gate checks if tests_passed == true. If false, the condition fails.

2. Step 2: Understand pipeline behavior on failed test gate

   When the condition fails, the pipeline stops and does not continue to the deploy stage.

3. Final Answer:

   Pipeline stops and does not proceed to deploy -> Option A

4. Quick Check:

   Failed test gate = pipeline stops [OK]

Hint: Failed test gate stops pipeline flow [OK]

Common Mistakes:

• Assuming pipeline skips tests
• Thinking pipeline restarts automatically
• Believing deploy runs despite failed tests

4. You have this pipeline snippet:

stages:
  - build
  - test
  - deploy

test_gate:
  stage: test
  condition: tests_passed
  action: continue_pipeline

But the pipeline always proceeds even when tests fail. What is the likely error?

medium

A. The condition should explicitly check for true, e.g., tests_passed == true

B. The deploy stage is missing

C. The build stage is not defined

D. The test gate action should be 'stop_pipeline' instead of 'continue_pipeline'

Solution

1. Step 1: Examine the test gate condition

   The condition uses tests_passed alone, which may not evaluate correctly if it's not a boolean.

2. Step 2: Correct the condition syntax

   Explicitly checking tests_passed == true ensures the gate only passes when tests truly pass.

3. Final Answer:

   The condition should explicitly check for true, e.g., tests_passed == true -> Option A

4. Quick Check:

   Explicit condition check fixes gate logic [OK]

Hint: Use explicit boolean checks in test gates [OK]

Common Mistakes:

• Ignoring condition syntax errors
• Assuming missing stages cause gate failure
• Confusing action names in test gates

5. A pipeline has these stages:

1. Build
2. Unit Test
3. Integration Test
4. Deploy

Test gates are set after Unit Test and Integration Test stages. If the Integration Test gate fails, what should happen to ensure safe deployment?

hard

A. Deploy only if Unit Test gate passed, ignoring Integration Test

B. Skip Integration Test and deploy anyway

C. Stop the pipeline and prevent deployment

D. Restart the pipeline from Build stage automatically

Solution

1. Step 1: Understand test gate purpose after Integration Test

   The gate ensures that integration tests pass before deployment to avoid faulty releases.

2. Step 2: Determine correct pipeline behavior on gate failure

   If the Integration Test gate fails, the pipeline must stop to prevent deploying broken software.

3. Final Answer:

   Stop the pipeline and prevent deployment -> Option C

4. Quick Check:

   Failing test gate = stop deployment [OK]

Hint: Failing integration tests block deployment [OK]

Common Mistakes:

• Ignoring integration test failures
• Assuming automatic pipeline restart
• Deploying despite failed test gates