Continuous Delivery testing in Testing Fundamentals - Cheat Sheet & Quick Revision

Recall & Review

beginner

What is Continuous Delivery (CD) in software development?

Continuous Delivery is a practice where code changes are automatically prepared for release to production. It ensures software can be reliably released at any time.

beginner

Why is testing important in Continuous Delivery?

Testing ensures that every code change is safe and does not break the software. It helps maintain quality while delivering updates quickly.

intermediate

Name two types of tests commonly automated in Continuous Delivery pipelines.

Unit tests and integration tests are commonly automated to quickly check small parts and their interactions.

intermediate

What role does a staging environment play in Continuous Delivery testing?

A staging environment mimics production where final tests run to catch issues before release, ensuring the software works as expected in real conditions.

beginner

How does Continuous Delivery testing help reduce risks in software releases?

By testing every change automatically and frequently, it catches bugs early, reduces surprises, and allows quick fixes before reaching users.

What is the main goal of Continuous Delivery testing?

A To delay software releases for more manual checks

B To write code without any tests

C To ensure code changes can be released safely at any time

D To avoid automation in testing

Continuous Delivery testing aims to make sure software updates are always ready and safe to release.

Which test type is usually the fastest to run in a Continuous Delivery pipeline?

A Unit tests

B End-to-end tests

C Manual exploratory tests

D Performance tests

Unit tests check small pieces of code and run quickly, making them ideal for fast feedback.

What environment is typically used to test software before production in Continuous Delivery?

A Staging environment

B Development environment

C Production environment

D Testing environment

The staging environment closely mimics production and is used for final testing.

Which of these is NOT a benefit of Continuous Delivery testing?

A Faster bug detection

B Reduced manual testing effort

C More reliable software releases

D Increased risk of broken releases

Continuous Delivery testing reduces risks, it does not increase them.

What is a common tool used to automate tests in Continuous Delivery?

A Manual test scripts

B CI/CD pipeline tools like Jenkins or GitHub Actions

C Email clients

D Word processors

CI/CD tools automate running tests and deploying code changes.

Explain how testing fits into the Continuous Delivery process and why it is important.

Think about how testing helps deliver software quickly without breaking it.

Describe the types of tests you would automate in a Continuous Delivery pipeline and their purpose.

Consider tests that check different levels of the software.

Practice

1. What is the main goal of Continuous Delivery testing in software development?

easy

A. To delay testing until after deployment

B. To write manual test cases for every feature

C. To ensure the software is always in a releasable state

D. To test only the user interface manually

Solution

1. Step 1: Understand Continuous Delivery testing purpose

   Continuous Delivery testing aims to keep software ready to release at any time by running automated tests frequently.

2. Step 2: Compare options with the goal

   Only To ensure the software is always in a releasable state matches this goal. Other options describe manual or delayed testing, which contradict Continuous Delivery principles.

3. Final Answer:

   To ensure the software is always in a releasable state -> Option C

4. Quick Check:

   Continuous Delivery = Always releasable software [OK]

Hint: Continuous Delivery means software is always ready to release [OK]

Common Mistakes:

• Thinking manual testing is the main focus
• Believing testing happens only after deployment
• Confusing Continuous Delivery with manual UI testing

2. Which of the following is the correct syntax to define a simple automated test step in a Continuous Delivery pipeline script?

easy

A. run_tests script pytest tests/

B. run_tests: script: pytest tests/

C. run_tests: pytest tests/

D. run_tests = pytest tests/

Solution

1. Step 1: Identify correct YAML syntax for pipeline steps

   Pipeline scripts often use YAML format where keys have colons and nested keys are indented, e.g., 'run_tests: script: pytest tests/'.

2. Step 2: Check each option's syntax

   run_tests: script: pytest tests/ uses correct YAML with colon and nested script key. Options A, C, and D miss colons or use invalid assignment.

3. Final Answer:

   run_tests: script: pytest tests/ -> Option B

4. Quick Check:

   YAML step syntax = run_tests: script: command [OK]

Hint: YAML keys need colons and proper indentation [OK]

Common Mistakes:

• Omitting colons in YAML keys
• Using equal signs instead of colons
• Not indenting nested keys properly

3. Consider this simplified test pipeline code snippet:

tests = ['unit', 'integration', 'e2e']
results = []
for test in tests:
    if test == 'integration':
        results.append('fail')
    else:
        results.append('pass')
print(results)

What will be the output after running this code?

medium

A. ['pass', 'fail', 'pass']

B. ['fail', 'fail', 'fail']

C. ['pass', 'pass', 'fail']

D. ['fail', 'pass', 'fail']

Solution

1. Step 1: Analyze the loop and conditions

   The list 'tests' has three items: 'unit', 'integration', 'e2e'. The loop checks if the test is 'integration' to append 'fail', else 'pass'.

2. Step 2: Determine results for each test

   'unit' -> not 'integration' -> 'pass'
   'integration' -> equals 'integration' -> 'fail'
   'e2e' -> not 'integration' -> 'pass'

3. Final Answer:

   ['pass', 'fail', 'pass'] -> Option A

4. Quick Check:

   Only 'integration' test fails [OK]

Hint: Only 'integration' test appends 'fail' [OK]

Common Mistakes:

• Assuming all tests pass
• Confusing the condition logic
• Mixing order of results

4. A Continuous Delivery pipeline script has this step:

test_step:
  script:
    - pytest --maxfail=1 --disable-warnings
    - echo "Tests completed"

However, the pipeline does not stop on the first test failure as expected. What is the likely cause?

medium

A. pytest does not support the --maxfail option

B. The --maxfail=1 option is invalid and ignored

C. The echo command causes the pipeline to stop early

D. The pipeline runs all script commands regardless of failure by default

Solution

1. Step 1: Understand pipeline script behavior

   In many CI/CD tools, all commands under 'script' run sequentially even if one fails, unless configured to stop on failure.

2. Step 2: Analyze the options and commands

   --maxfail=1 is a valid pytest option to stop pytest early, but the pipeline still runs the next command 'echo'. So the pipeline itself does not stop on failure automatically.

3. Final Answer:

   The pipeline runs all script commands regardless of failure by default -> Option D

4. Quick Check:

   Pipeline runs all script lines unless configured to stop [OK]

Hint: Pipeline runs all script commands unless told to stop [OK]

Common Mistakes:

• Thinking pytest controls pipeline flow
• Believing echo causes failure
• Assuming invalid option causes issue

5. In a Continuous Delivery pipeline, you want to run tests only on changed files to speed up feedback. Which approach best achieves this?

hard

A. Use a script to detect changed files and run tests related to those files

B. Run all tests every time regardless of changes

C. Manually select tests to run after each commit

D. Disable automated tests and rely on manual testing

Solution

1. Step 1: Understand test optimization in Continuous Delivery

   Running all tests every time can be slow. Detecting changed files and running only related tests speeds feedback.

2. Step 2: Evaluate options for automation and speed

   Use a script to detect changed files and run tests related to those files automates detection and selective testing, fitting Continuous Delivery goals. Options B, C, and D are inefficient or manual.

3. Final Answer:

   Use a script to detect changed files and run tests related to those files -> Option A

4. Quick Check:

   Selective testing on changes = faster feedback [OK]

Hint: Run tests only for changed files to save time [OK]

Common Mistakes:

• Running all tests every time without optimization
• Relying on manual test selection
• Skipping automated tests entirely