Why CI/CD integrates testing into delivery in Testing Fundamentals - Framework Benefits

Framework Mode - Why CI/CD integrates testing into delivery

Folder Structure of a CI/CD Testing Project

ci-cd-testing-project/
├── src/
│   ├── main/
│   │   └── app-code/
│   └── test/
│       ├── unit/
│       ├── integration/
│       └── e2e/
├── scripts/
│   └── ci-cd-pipelines/
│       ├── build.yml
│       ├── test.yml
│       └── deploy.yml
├── config/
│   ├── environments/
│   │   ├── dev.env
│   │   ├── staging.env
│   │   └── prod.env
│   └── test-config.yml
├── reports/
│   └── test-results/
├── docs/
└── README.md

Test Framework Layers in CI/CD

• Source Code Layer: Application code and test code (unit, integration, end-to-end).
• Test Execution Layer: Automated tests run by CI server on code changes.
• Configuration Layer: Environment variables, test data, credentials managed securely.
• Reporting Layer: Test results collected and displayed for quick feedback.
• Pipeline Layer: Scripts and workflows that build, test, and deploy automatically.

Configuration Patterns for CI/CD Testing

• Environment Files: Separate files for dev, staging, and production to manage settings.
• Secrets Management: Use secure vaults or encrypted storage for credentials.
• Parameterization: Pass browser types, test suites, or feature flags as parameters in pipeline scripts.
• Version Control: Store all configs and scripts in the same repository for traceability.

Test Reporting and CI/CD Integration

• Automated Test Reports: Generate HTML or XML reports after test runs for easy review.
• CI Dashboard: Display pass/fail status and trends in the CI tool (e.g., Jenkins, GitHub Actions).
• Notifications: Send alerts via email or chat when tests fail to fix issues quickly.
• Gatekeeping: Block deployment if critical tests fail to ensure quality.

Best Practices for Integrating Testing into CI/CD

• Fail Fast: Run quick tests first to catch errors early.
• Automate Everything: No manual steps in build, test, or deploy.
• Isolate Tests: Keep tests independent to avoid false failures.
• Maintain Test Data: Use stable and realistic data for reliable results.
• Continuous Feedback: Provide immediate results to developers for fast fixes.

Self Check

Where in this folder structure would you add a new automated test script for a feature's end-to-end test?

Key Result

CI/CD integrates automated testing to catch issues early and ensure quality before delivery.

Practice

1. Why is testing integrated into the CI/CD pipeline?

easy

A. To avoid writing code

B. To slow down the delivery process

C. To increase manual work for developers

D. To find problems early and improve software quality

Solution

1. Step 1: Understand the purpose of CI/CD testing

   Testing in CI/CD helps catch bugs early before software reaches users.

2. Step 2: Recognize the benefit of early problem detection

   Finding issues early improves software quality and reduces costly fixes later.

3. Final Answer:

   To find problems early and improve software quality -> Option D

4. Quick Check:

   Early testing = better quality [OK]

Hint: Testing in CI/CD finds bugs early to improve quality [OK]

Common Mistakes:

• Thinking testing slows delivery
• Believing testing is optional
• Assuming testing increases manual work

2. Which of the following is the correct way to add a test stage in a CI/CD YAML pipeline?

easy

A. steps:
- build
- deploy
- test

B. stages:
- build
- test
- deploy

C. jobs:
- deploy
- test
- build

D. phases:
- test
- build
- deploy

Solution

1. Step 1: Identify correct CI/CD pipeline syntax

   Most CI/CD tools use 'stages' to define order: build, test, deploy.

2. Step 2: Check the order of stages

   Testing should come after build and before deploy for proper flow.

3. Final Answer:

   stages: - build - test - deploy -> Option B

4. Quick Check:

   Stages order: build, test, deploy [OK]

Hint: Stages define order: build, test, then deploy [OK]

Common Mistakes:

• Mixing steps and stages keywords
• Placing test after deploy
• Using unsupported keywords like phases

3. In a CI/CD pipeline, why is it important to run automated tests before deployment?

easy

A. To increase manual testing effort

B. To delay the deployment process intentionally

C. To catch bugs early and prevent faulty code from reaching users

D. To skip code reviews

Solution

1. Step 1: Understand the role of automated tests in CI/CD

   Automated tests verify code correctness quickly and reliably before deployment.

2. Step 2: Recognize the impact on software quality and user experience

   Running tests early prevents bugs from reaching users, improving reliability.

3. Final Answer:

   To catch bugs early and prevent faulty code from reaching users -> Option C

4. Quick Check:

   Tests before deploy = fewer bugs in production [OK]

Hint: Tests before deploy catch bugs early [OK]

Common Mistakes:

• Thinking tests delay deployment on purpose
• Confusing automated tests with manual effort
• Believing tests replace code reviews

4. Consider this simplified CI/CD pipeline snippet:

stages:
  - build
  - test

build_job:
  stage: build
  script:
    - echo "Building..."
test_job:
  stage: test
  script:
    - exit 1

What will happen when this pipeline runs?

medium

A. The build job runs, then the test job fails, stopping the pipeline

B. Both build and test jobs succeed

C. The test job runs before the build job

D. The pipeline ignores the test job failure and continues

Solution

1. Step 1: Analyze job order by stages

   Build stage runs first, then test stage runs after build completes.

2. Step 2: Check test job script

   Test job script exits with code 1, which means failure.

3. Final Answer:

   The build job runs, then the test job fails, stopping the pipeline -> Option A

4. Quick Check:

   Failing test stops pipeline [OK]

Hint: Exit code 1 means test failure stops pipeline [OK]

Common Mistakes:

• Assuming test runs before build
• Thinking pipeline ignores failures
• Believing all jobs always succeed

5. A CI/CD pipeline has this test stage:

test_job:
  stage: test
  script:
    - pytest tests/
  only:
    - main
    - develop

Why might tests not run on a feature branch?

medium

A. Because the 'only' keyword limits runs to 'main' and 'develop' branches

B. Because pytest is not installed

C. Because the test stage is missing

D. Because the script syntax is incorrect

Solution

1. Step 1: Understand 'only' keyword effect

   'only' restricts job runs to specified branches: main and develop.

2. Step 2: Check branch context

   Feature branches are not listed, so tests won't run there.

3. Final Answer:

   Because the 'only' keyword limits runs to 'main' and 'develop' branches -> Option A

4. Quick Check:

   'only' limits branches [OK]

Hint: 'only' controls which branches run jobs [OK]

Common Mistakes:

• Assuming tests run on all branches
• Blaming missing pytest without checking config
• Ignoring branch filters in pipeline