Why different testing levels catch different bugs in Testing Fundamentals - Framework Benefits

Framework Mode - Why different testing levels catch different bugs

Folder Structure of a Typical Test Project

  project-root/
  ├── unit_tests/
  │   ├── test_module1.py
  │   └── test_module2.py
  ├── integration_tests/
  │   ├── test_integration_featureA.py
  │   └── test_integration_featureB.py
  ├── system_tests/
  │   ├── test_system_end_to_end.py
  │   └── test_system_performance.py
  ├── acceptance_tests/
  │   └── test_acceptance_criteria.py
  ├── fixtures/
  │   └── common_fixtures.py
  ├── utils/
  │   └── helpers.py
  ├── config/
  │   └── test_config.yaml
  └── README.md
  

Test Framework Layers

• Unit Tests Layer: Tests small pieces of code like functions or classes. Catches bugs in logic or calculations early.
• Integration Tests Layer: Tests how different parts work together. Finds bugs in data flow or interface mismatches.
• System Tests Layer: Tests the whole application as a user would. Catches bugs in workflows, performance, and environment issues.
• Acceptance Tests Layer: Validates the app meets business needs and user requirements. Finds missing features or incorrect behavior.
• Utilities Layer: Helper functions and common code used by tests.
• Configuration Layer: Settings for environments, browsers, credentials, and test data.

Configuration Patterns

Use separate config files or environment variables to manage:

• Environments: dev, test, staging, production URLs and settings.
• Browsers/Platforms: Specify which browsers or devices to run tests on.
• Credentials: Store test user names and passwords securely.
• Test Data: Use fixtures or data files to provide input data for tests.

Example: test_config.yaml with environment URLs and credentials.

Test Reporting and CI/CD Integration

• Generate clear test reports showing passed and failed tests with details.
• Integrate with CI/CD pipelines to run tests automatically on code changes.
• Use tools like Jenkins, GitHub Actions, or GitLab CI to trigger tests and collect results.
• Send notifications on test failures to developers and testers.

Framework Design Principles

1. Layered Testing: Organize tests by levels to catch bugs early and in context.
2. Clear Separation: Keep unit, integration, system, and acceptance tests in separate folders.
3. Reusable Fixtures: Use common setup code to avoid duplication and keep tests clean.
4. Configurable Environments: Allow easy switching between test environments without code changes.
5. Automated Reporting: Provide fast feedback with detailed reports integrated into development workflow.

Self Check Question

Where in this folder structure would you add a new test that checks if two modules work correctly together?

Key Result

Organize tests in layers to catch different bugs early and clearly.

Practice

1. Which testing level focuses on checking individual parts of the software separately?

easy

A. Acceptance Testing

B. System Testing

C. Integration Testing

D. Unit Testing

Solution

1. Step 1: Understand testing levels

   Unit testing checks small parts or units of code individually.

2. Step 2: Compare options to definitions

   System testing checks the whole system, integration testing checks combined parts, acceptance testing checks if requirements are met.

3. Final Answer:

   Unit Testing -> Option D

4. Quick Check:

   Individual parts = Unit Testing [OK]

Hint: Unit testing tests small parts alone [OK]

Common Mistakes:

• Confusing integration testing with unit testing
• Thinking system testing tests parts separately
• Mixing acceptance testing with unit testing

2. Which of the following is the correct order of testing levels from smallest to largest scope?

easy

A. System Testing -> Integration Testing -> Unit Testing

B. Unit Testing -> Integration Testing -> System Testing

C. Integration Testing -> Unit Testing -> System Testing

D. Acceptance Testing -> System Testing -> Unit Testing

Solution

1. Step 1: Identify scope sizes

   Unit testing checks smallest parts, integration testing checks combined parts, system testing checks whole system.

2. Step 2: Arrange from smallest to largest

   Correct order is Unit Testing, then Integration Testing, then System Testing.

3. Final Answer:

   Unit Testing -> Integration Testing -> System Testing -> Option B

4. Quick Check:

   Smallest to largest = A [OK]

Hint: Remember: Unit, Integration, System order [OK]

Common Mistakes:

• Reversing the order of testing levels
• Confusing integration with system testing
• Including acceptance testing in the wrong place

3. Consider this scenario: A bug appears only when two modules interact but not in individual modules. Which testing level is most likely to catch this bug?

medium

A. Integration Testing

B. Unit Testing

C. System Testing

D. Acceptance Testing

Solution

1. Step 1: Analyze bug context

   The bug appears only when two modules interact, so it is related to combined parts.

2. Step 2: Match testing level to bug type

   Integration testing focuses on testing combined modules and their interactions.

3. Final Answer:

   Integration Testing -> Option A

4. Quick Check:

   Bug in combined modules = Integration Testing [OK]

Hint: Bugs in combined parts? Think Integration Testing [OK]

Common Mistakes:

• Choosing unit testing for interaction bugs
• Confusing system testing with integration testing
• Assuming acceptance testing finds module interaction bugs

4. A tester wrote a test that checks the entire software system but missed testing how modules work together. What testing level did they likely skip?

medium

A. Integration Testing

B. Unit Testing

C. Acceptance Testing

D. System Testing

Solution

1. Step 1: Understand the test scope

   The tester tested the whole system but did not check module interactions.

2. Step 2: Identify missing testing level

   Integration testing specifically checks how modules work together, so it was likely skipped.

3. Final Answer:

   Integration Testing -> Option A

4. Quick Check:

   Missed module interaction = Integration Testing [OK]

Hint: Check module interactions with Integration Testing [OK]

Common Mistakes:

• Confusing system testing with integration testing
• Assuming unit testing covers module interactions
• Thinking acceptance testing covers module integration

5. A software project has a bug that only appears when the full system runs with real user data, but not during unit or integration tests. Which testing level is best suited to find this bug?

hard

A. Unit Testing

B. Integration Testing

C. System Testing

D. Acceptance Testing

Solution

1. Step 1: Analyze bug conditions

   The bug appears only when the full system runs with real user data, not in smaller tests.

2. Step 2: Match testing level to scenario

   System testing checks the entire system working together, often with realistic data, so it is best to find such bugs.

3. Final Answer:

   System Testing -> Option C

4. Quick Check:

   Full system with real data = System Testing [OK]

Hint: Full system bugs? Use System Testing [OK]

Common Mistakes:

• Choosing unit or integration testing for full system bugs
• Confusing acceptance testing with system testing
• Ignoring the importance of real user data in tests