Test automation pyramid in Testing Fundamentals - Framework Patterns

Framework Mode - Test automation pyramid

Test Automation Pyramid: Folder Structure

  test-automation-project/
  ├── unit/
  │   ├── test_module1.py
  │   └── test_module2.py
  ├── integration/
  │   ├── test_integration1.py
  │   └── test_integration2.py
  ├── e2e/
  │   ├── test_login_flow.py
  │   └── test_checkout_flow.py
  ├── utils/
  │   ├── helpers.py
  │   └── fixtures.py
  ├── config/
  │   ├── config.yaml
  │   └── env_variables.py
  ├── reports/
  │   └── latest_report.html
  └── README.md
  

Test Automation Pyramid: Framework Layers

• Unit Tests Layer: Small, fast tests for individual functions or methods. Usually located in unit/.
• Integration Tests Layer: Tests that check how different parts work together. Found in integration/.
• End-to-End (E2E) Tests Layer: Tests that simulate real user scenarios from start to finish. Stored in e2e/.
• Utilities Layer: Helper functions, test data, and fixtures to support tests. Located in utils/.
• Configuration Layer: Settings for environments, credentials, and test parameters. Found in config/.

Configuration Patterns

Use a config.yaml or similar file to store environment URLs, credentials, and browser settings.

Example config.yaml content:

  environments:
    dev:
      url: "https://dev.example.com"
      browser: "chrome"
    staging:
      url: "https://staging.example.com"
      browser: "firefox"
  credentials:
    username: "testuser"
    password: "password123"
  

Load this config in tests to run against different environments without changing code.

Test Reporting and CI/CD Integration

• Generate HTML or XML reports after test runs, saved in reports/.
• Integrate with CI/CD tools (like Jenkins, GitHub Actions) to run tests automatically on code changes.
• Fail the build if critical tests fail, ensuring quality gates.
• Send test reports via email or messaging tools for team visibility.

Best Practices for Test Automation Pyramid

1. Test Fast at the Bottom: Write many unit tests because they run quickly and catch bugs early.
2. Limit E2E Tests: Keep end-to-end tests few and focused since they are slower and more fragile.
3. Clear Separation: Organize tests by type (unit, integration, e2e) to keep framework clean and maintainable.
4. Reusable Utilities: Use helper functions and fixtures to avoid repeating code across tests.
5. Configurable Environments: Use config files to easily switch between testing environments without code changes.

Self Check

Where in this folder structure would you add a new test that checks how the login page works with the database?

Hint: It is not a simple unit test, but it is not a full user flow either.

Key Result

Organize tests in layers: many fast unit tests at the base, fewer integration tests in the middle, and few slow end-to-end tests at the top.

Practice

1. What is the main idea behind the test automation pyramid?

easy

A. Write mostly UI tests because they cover all features

B. Write many fast unit tests, fewer integration tests, and few slow UI tests

C. Write only manual tests to save automation effort

D. Write equal numbers of unit, integration, and UI tests

Solution

1. Step 1: Understand test types and speed

   Unit tests are fast and cheap, integration tests are slower, UI tests are slowest and costly.

2. Step 2: Apply pyramid principle

   The pyramid suggests many unit tests, fewer integration tests, and few UI tests to catch bugs early and keep tests efficient.

3. Final Answer:

   Write many fast unit tests, fewer integration tests, and few slow UI tests -> Option B

4. Quick Check:

   Test automation pyramid = many unit, fewer integration, few UI tests [OK]

Hint: Remember: fast tests at bottom, slow tests at top [OK]

Common Mistakes:

• Thinking UI tests should be the majority
• Believing manual tests replace automation
• Assuming all test types should be equal

2. Which of the following correctly orders test types from fastest to slowest in the test automation pyramid?

easy

A. UI tests, integration tests, unit tests

B. Integration tests, unit tests, UI tests

C. Integration tests, UI tests, unit tests

D. Unit tests, integration tests, UI tests

Solution

1. Step 1: Recall test speed hierarchy

   Unit tests run fastest because they test small code parts, integration tests are slower, UI tests are slowest due to full system interaction.

2. Step 2: Match order from fastest to slowest

   The correct order is unit tests first, then integration tests, then UI tests last.

3. Final Answer:

   Unit tests, integration tests, UI tests -> Option D

4. Quick Check:

   Speed order = unit < integration < UI [OK]

Hint: Fastest tests are smallest scope: unit first [OK]

Common Mistakes:

• Confusing UI tests as fastest
• Mixing integration and unit test order
• Ignoring test execution speed differences

3. Consider this test suite distribution: 100 unit tests, 30 integration tests, and 5 UI tests. Which statement best describes this setup according to the test automation pyramid?

medium

A. It has too few integration tests compared to UI tests

B. It has too many UI tests compared to unit tests

C. It follows the pyramid by having many unit tests and few UI tests

D. It violates the pyramid by having equal test counts

Solution

1. Step 1: Analyze test counts

   There are many unit tests (100), fewer integration tests (30), and very few UI tests (5).

2. Step 2: Compare with pyramid principle

   The pyramid recommends many unit tests, fewer integration tests, and few UI tests, matching this distribution.

3. Final Answer:

   It follows the pyramid by having many unit tests and few UI tests -> Option C

4. Quick Check:

   Test counts match pyramid shape [OK]

Hint: More unit tests, less UI tests = pyramid shape [OK]

Common Mistakes:

• Thinking 5 UI tests is too many
• Confusing integration test count importance
• Assuming equal test numbers are correct

4. A team wrote mostly UI tests and very few unit tests. What is the main problem with this approach according to the test automation pyramid?

medium

A. UI tests are slow and costly, causing slow feedback and maintenance issues

B. Unit tests are unnecessary if UI tests exist

C. Integration tests will automatically cover all bugs

D. Manual testing is better than UI tests

Solution

1. Step 1: Understand test cost and speed

   UI tests are slow to run and expensive to maintain compared to unit tests.

2. Step 2: Identify pyramid violation

   Having mostly UI tests means slow feedback and higher maintenance, which the pyramid warns against.

3. Final Answer:

   UI tests are slow and costly, causing slow feedback and maintenance issues -> Option A

4. Quick Check:

   Too many UI tests = slow, costly [OK]

Hint: Too many UI tests slow feedback [OK]

Common Mistakes:

• Ignoring unit test importance
• Assuming integration tests cover all bugs
• Confusing manual and automated UI tests

5. You want to improve your test suite following the test automation pyramid. Currently, you have 20 UI tests, 50 integration tests, and 10 unit tests. What is the best action to align with the pyramid?

hard

A. Increase unit tests significantly, reduce UI tests, and keep integration tests moderate

B. Increase UI tests to cover more features, keep others same

C. Remove unit tests and focus on integration and UI tests

D. Keep all test counts the same but run UI tests more often

Solution

1. Step 1: Analyze current test distribution

   There are few unit tests (10), many integration tests (50), and many UI tests (20), which is opposite of pyramid advice.

2. Step 2: Apply pyramid principle for improvement

   The pyramid suggests many unit tests, fewer integration tests, and few UI tests to catch bugs early and keep tests efficient.

3. Final Answer:

   Increase unit tests significantly, reduce UI tests, and keep integration tests moderate -> Option A

4. Quick Check:

   More unit, fewer UI tests = pyramid aligned [OK]

Hint: Boost unit tests, cut UI tests for pyramid fit [OK]

Common Mistakes:

• Adding more UI tests instead of unit tests
• Removing unit tests thinking they are less useful
• Ignoring test speed and cost differences