Continuous testing in CI/CD in Testing Fundamentals - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

What is the main purpose of continuous testing in a CI/CD pipeline?

Continuous testing is a key part of CI/CD. What is its main goal?

A To run tests only after the entire software is deployed to production

B To automatically run tests early and often to catch bugs quickly

C To manually check code quality before merging changes

D To delay testing until all features are complete

💡 Hint

Think about when tests should run to find problems fast.

Continuous testing means running automated tests frequently during development to find bugs early, not waiting until deployment.

💻 Command Output

intermediate

What is the output of this Jenkins pipeline stage running tests?

Given this Jenkinsfile snippet, what will be the output if tests fail?

Testing Fundamentals

stage('Test') {
  steps {
    sh 'pytest tests/'
  }
}

A The pipeline stops and marks the build as failed

B The pipeline continues to deploy even if tests fail

C The pipeline ignores test results and passes

D The pipeline retries tests indefinitely

💡 Hint

How does Jenkins treat shell command failures by default?

In Jenkins, if a shell command like 'pytest' fails (non-zero exit), the pipeline stage fails and stops further steps.

🔀 Workflow

advanced

Order the steps in a typical continuous testing workflow in CI/CD

Arrange these steps in the correct order for continuous testing in a CI/CD pipeline.

A 2,3,1,4

B 3,2,1,4

C 2,1,3,4

D 1,2,3,4

💡 Hint

Think about what happens first: code push or pipeline trigger?

The developer pushes code, then the CI server triggers the pipeline, which runs tests, then reports results.

❓ Troubleshoot

advanced

Why might tests not run automatically in a CI/CD pipeline?

You set up a CI/CD pipeline but notice tests are not running after code pushes. What could cause this?

A The code repository is empty

B The CI/CD server is offline permanently

C The test commands are missing or incorrectly specified in the pipeline config

D The developer forgot to write any tests

💡 Hint

Check the pipeline configuration for test steps.

If the pipeline config does not include commands to run tests, tests will not execute automatically.

✅ Best Practice

expert

Which practice best improves test reliability in continuous testing?

In continuous testing, which practice helps ensure tests give consistent results?

A Running tests only on developer machines

B Running tests manually after deployment

C Skipping tests that sometimes fail randomly

D Using isolated test environments that reset before each run

💡 Hint

Think about how to avoid test failures caused by leftover data or state.

Isolated, clean test environments prevent interference from previous runs, improving test reliability.

Practice

1. What is the main purpose of continuous testing in a CI/CD pipeline?

easy

A. To automatically run tests on every code change

B. To manually test software after deployment

C. To delay testing until the final release

D. To only test code on the developer's machine

Solution

1. Step 1: Understand continuous testing concept

   Continuous testing means running tests automatically during the CI/CD process.

2. Step 2: Identify the purpose in CI/CD

   Its goal is to catch bugs early by testing every code change automatically.

3. Final Answer:

   To automatically run tests on every code change -> Option A

4. Quick Check:

   Continuous testing = automatic tests on code changes [OK]

Hint: Continuous testing means tests run automatically every change [OK]

Common Mistakes:

• Thinking testing is manual only
• Believing tests run after deployment
• Assuming testing happens only once at release

2. Which of the following is the correct syntax to run tests in a simple CI pipeline using a shell command?

easy

A. sh tests.sh

B. run tests.sh

C. execute tests.sh

D. start tests.sh

Solution

1. Step 1: Recall shell command syntax

   To run a shell script, the correct command is 'sh scriptname.sh'.

2. Step 2: Check each option

   sh tests.sh uses 'sh tests.sh' which correctly runs the script. Others are invalid shell commands.

3. Final Answer:

   sh tests.sh -> Option A

4. Quick Check:

   Shell script run = sh script.sh [OK]

Hint: Use 'sh script.sh' to run shell scripts in pipelines [OK]

Common Mistakes:

• Using 'run' or 'start' which are not shell commands
• Forgetting to use 'sh' before script name
• Confusing shell commands with other languages

3. Given this CI pipeline snippet:

steps:
  - name: Run tests
    run: |
      echo "Starting tests"
      pytest tests/
      echo "Tests done"

What will be the output if all tests pass?

medium

A. Starting tests\nTests failed

B. Starting tests\npytest: command not found

C. Starting tests\nTests done

D. Tests done\nStarting tests

Solution

1. Step 1: Analyze the commands in order

   The pipeline runs 'echo "Starting tests"', then 'pytest tests/', then 'echo "Tests done"'.

2. Step 2: Understand output if tests pass

   If pytest runs successfully, it outputs test results but here we focus on echo commands output. So output includes 'Starting tests' then 'Tests done'.

3. Final Answer:

   Starting tests\nTests done -> Option C

4. Quick Check:

   Echo commands print start and done messages [OK]

Hint: Echo commands print messages before and after tests [OK]

Common Mistakes:

• Assuming pytest output is shown instead of echo
• Mixing order of printed lines
• Thinking tests failed without error

4. In a CI pipeline, the test step fails with error: sh: pytest: command not found. What is the most likely cause?

medium

A. The test script has a syntax error

B. The pipeline skipped the test step

C. The tests directory is missing

D. pytest is not installed in the pipeline environment

Solution

1. Step 1: Interpret the error message

   The error 'sh: pytest: command not found' means the shell cannot find the pytest command.

2. Step 2: Identify cause in CI environment

   This usually means pytest is not installed or not in PATH in the pipeline environment.

3. Final Answer:

   pytest is not installed in the pipeline environment -> Option D

4. Quick Check:

   Command not found = missing installation [OK]

Hint: Command not found means tool missing or not installed [OK]

Common Mistakes:

• Assuming syntax error causes command not found
• Thinking missing tests directory causes this error
• Believing pipeline skipped the step without error

5. You want to add continuous testing to your CI/CD pipeline so tests run only if code changes affect the src/ directory. Which approach best achieves this?

hard

A. Run tests on every pipeline run regardless of changes

B. Configure the pipeline to run tests only when files in src/ change

C. Run tests only after deployment to production

D. Manually trigger tests after code review

Solution

1. Step 1: Understand conditional testing in CI/CD

   To save time, tests should run only when relevant code changes occur.

2. Step 2: Apply file path filters in pipeline configuration

   Configuring the pipeline to detect changes in src/ and run tests only then is the best approach.

3. Final Answer:

   Configure the pipeline to run tests only when files in src/ change -> Option B

4. Quick Check:

   Run tests conditionally based on changed files [OK]

Hint: Run tests only if relevant files change to save time [OK]

Common Mistakes:

• Running tests every time wastes resources
• Testing only after deployment delays bug detection
• Manual triggers reduce automation benefits