Why CI/CD integrates testing into delivery in Testing Fundamentals - Automation Benefits in Action

Verify automated tests run during CI/CD pipeline

Preconditions (3)

Step 1: Push a code change to the repository branch that triggers the CI/CD pipeline

Step 2: Observe the CI/CD pipeline execution start

Step 3: Verify that the automated tests are executed as part of the pipeline

Step 4: Check the test results in the pipeline logs or reports

✅ Expected Result: The CI/CD pipeline runs the automated tests automatically after code changes, and the test results are visible in the pipeline report indicating pass or fail status.

Automation Requirements - pytest with pytest-cov for coverage

Assertions Needed:

Verify test functions run without errors

Assert expected outputs or states in test cases

Confirm test coverage reports are generated

Best Practices:

Use fixtures for setup and teardown

Write small, independent test functions

Use descriptive test names

Integrate tests into CI/CD pipeline configuration

Use explicit assertions for clarity

Automated Solution

Testing Fundamentals

import pytest

# Sample function to test
def add(a, b):
    return a + b

# Test function for add
def test_add_positive_numbers():
    assert add(2, 3) == 5

def test_add_negative_numbers():
    assert add(-1, -1) == -2

def test_add_zero():
    assert add(0, 0) == 0

if __name__ == '__main__':
    pytest.main(['-v', '--cov=.', '--cov-report=term-missing'])

This script defines a simple function add and three test functions to verify its behavior with positive numbers, negative numbers, and zero.

We use pytest as the test framework because it is easy to write and read tests. The assert statements check that the function returns expected results.

The pytest.main() call runs the tests with verbose output and coverage reporting, which helps verify that tests are executed and code coverage is measured.

In a CI/CD pipeline, this script would be triggered automatically after code changes, ensuring tests run every time code is delivered.

Common Mistakes - 4 Pitfalls

Not using explicit assertions in test functions

Writing large test functions that test multiple things

{'mistake': 'Not integrating tests into the CI/CD pipeline configuration', 'why_bad': "Tests won't run automatically on code changes, defeating the purpose of CI/CD testing.", 'correct_approach': 'Configure the CI/CD pipeline to run tests on every code push or pull request.'}

Hardcoding test data inside tests without flexibility

Bonus Challenge

Now add data-driven testing with 3 different input pairs for the add function

Show Hint

Practice

1. Why is testing integrated into the CI/CD pipeline?

easy

A. To avoid writing code

B. To slow down the delivery process

C. To increase manual work for developers

D. To find problems early and improve software quality

Solution

1. Step 1: Understand the purpose of CI/CD testing

   Testing in CI/CD helps catch bugs early before software reaches users.

2. Step 2: Recognize the benefit of early problem detection

   Finding issues early improves software quality and reduces costly fixes later.

3. Final Answer:

   To find problems early and improve software quality -> Option D

4. Quick Check:

   Early testing = better quality [OK]

Hint: Testing in CI/CD finds bugs early to improve quality [OK]

Common Mistakes:

• Thinking testing slows delivery
• Believing testing is optional
• Assuming testing increases manual work

2. Which of the following is the correct way to add a test stage in a CI/CD YAML pipeline?

easy

A. steps:
- build
- deploy
- test

B. stages:
- build
- test
- deploy

C. jobs:
- deploy
- test
- build

D. phases:
- test
- build
- deploy

Solution

1. Step 1: Identify correct CI/CD pipeline syntax

   Most CI/CD tools use 'stages' to define order: build, test, deploy.

2. Step 2: Check the order of stages

   Testing should come after build and before deploy for proper flow.

3. Final Answer:

   stages: - build - test - deploy -> Option B

4. Quick Check:

   Stages order: build, test, deploy [OK]

Hint: Stages define order: build, test, then deploy [OK]

Common Mistakes:

• Mixing steps and stages keywords
• Placing test after deploy
• Using unsupported keywords like phases

3. In a CI/CD pipeline, why is it important to run automated tests before deployment?

easy

A. To increase manual testing effort

B. To delay the deployment process intentionally

C. To catch bugs early and prevent faulty code from reaching users

D. To skip code reviews

Solution

1. Step 1: Understand the role of automated tests in CI/CD

   Automated tests verify code correctness quickly and reliably before deployment.

2. Step 2: Recognize the impact on software quality and user experience

   Running tests early prevents bugs from reaching users, improving reliability.

3. Final Answer:

   To catch bugs early and prevent faulty code from reaching users -> Option C

4. Quick Check:

   Tests before deploy = fewer bugs in production [OK]

Hint: Tests before deploy catch bugs early [OK]

Common Mistakes:

• Thinking tests delay deployment on purpose
• Confusing automated tests with manual effort
• Believing tests replace code reviews

4. Consider this simplified CI/CD pipeline snippet:

stages:
  - build
  - test

build_job:
  stage: build
  script:
    - echo "Building..."
test_job:
  stage: test
  script:
    - exit 1

What will happen when this pipeline runs?

medium

A. The build job runs, then the test job fails, stopping the pipeline

B. Both build and test jobs succeed

C. The test job runs before the build job

D. The pipeline ignores the test job failure and continues

Solution

1. Step 1: Analyze job order by stages

   Build stage runs first, then test stage runs after build completes.

2. Step 2: Check test job script

   Test job script exits with code 1, which means failure.

3. Final Answer:

   The build job runs, then the test job fails, stopping the pipeline -> Option A

4. Quick Check:

   Failing test stops pipeline [OK]

Hint: Exit code 1 means test failure stops pipeline [OK]

Common Mistakes:

• Assuming test runs before build
• Thinking pipeline ignores failures
• Believing all jobs always succeed

5. A CI/CD pipeline has this test stage:

test_job:
  stage: test
  script:
    - pytest tests/
  only:
    - main
    - develop

Why might tests not run on a feature branch?

medium

A. Because the 'only' keyword limits runs to 'main' and 'develop' branches

B. Because pytest is not installed

C. Because the test stage is missing

D. Because the script syntax is incorrect

Solution

1. Step 1: Understand 'only' keyword effect

   'only' restricts job runs to specified branches: main and develop.

2. Step 2: Check branch context

   Feature branches are not listed, so tests won't run there.

3. Final Answer:

   Because the 'only' keyword limits runs to 'main' and 'develop' branches -> Option A

4. Quick Check:

   'only' limits branches [OK]

Hint: 'only' controls which branches run jobs [OK]

Common Mistakes:

• Assuming tests run on all branches
• Blaming missing pytest without checking config
• Ignoring branch filters in pipeline