When to automate vs manual test in Testing Fundamentals - Framework Approaches Compared

Framework Mode - When to automate vs manual test

Folder Structure for Test Automation Project

project-root/
├── tests/                 # Automated test scripts
│   ├── regression/        # Automated regression tests
│   ├── smoke/             # Automated smoke tests
│   └── exploratory/       # Manual test cases documentation
├── pages/                 # Page objects or UI abstractions
├── utils/                 # Helper functions and utilities
├── config/                # Configuration files (env, credentials)
├── reports/               # Test execution reports
└── ci/                    # CI/CD pipeline scripts

This structure supports both automated and manual testing efforts.

Test Framework Layers

• Test Scripts Layer: Automated tests for repeatable scenarios (e.g., regression, smoke).
• Manual Test Documentation: Test cases and steps for manual exploratory or usability tests.
• Page Objects / UI Abstractions: Encapsulate UI elements for automation reuse.
• Utilities: Helpers for data setup, API calls, waits, logging.
• Configuration: Environment settings, credentials, browser choices.
• Reporting: Collect and present test results for both manual and automated tests.

Configuration Patterns

• Environment Files: Separate configs for dev, test, staging, production.
• Browser Settings: Choose browsers for automation runs (e.g., Chrome, Firefox).
• Credentials Management: Secure storage for usernames and passwords.
• Test Type Flags: Enable or disable automation vs manual test runs.

Test Reporting and CI/CD Integration

• Automated Test Reports: Generate pass/fail summaries with screenshots and logs.
• Manual Test Results: Document manual test outcomes in test management tools or spreadsheets.
• CI/CD Pipelines: Run automated tests on code changes and report results automatically.
• Notifications: Alert team on test failures or manual test completion.

Framework Design Principles for When to Automate vs Manual Test

1. Automate repetitive and stable tests: Focus automation on tests that run often and have predictable results.
2. Manual testing for exploratory and usability: Use manual tests when human judgment or creativity is needed.
3. Keep manual and automated tests documented: Clear test cases help decide what to automate later.
4. Maintain automation scripts regularly: Update automated tests as the application changes to avoid false failures.
5. Use configuration to switch test modes: Easily enable or disable automation or manual test runs based on project needs.

Self-Check Question

Where in this folder structure would you add documentation for manual exploratory test cases?

Key Result

Use automation for repeatable tests and manual testing for exploratory or usability scenarios.

Practice

1. Which type of testing is best suited for repetitive tasks that rarely change?

easy

A. Manual testing

B. Automated testing

C. Exploratory testing

D. Ad-hoc testing

Solution

1. Step 1: Understand the nature of repetitive tasks

   Repetitive tasks are those that happen often and have stable expected results.

2. Step 2: Match testing type to task nature

   Automated testing is ideal for repetitive and stable tasks because it saves time and reduces human error.

3. Final Answer:

   Automated testing -> Option B

4. Quick Check:

   Repetitive tasks = Automated testing [OK]

Hint: Automate stable, repetitive tests to save time [OK]

Common Mistakes:

• Choosing manual testing for repetitive tasks wastes time
• Confusing exploratory testing with repetitive testing
• Thinking ad-hoc testing fits repetitive tasks

2. Which of the following is the correct reason to choose manual testing over automation?

easy

A. When tests require human judgment and intuition

B. When tests are simple and can be scripted easily

C. When tests need to run overnight without interruption

D. When tests are repetitive and stable

Solution

1. Step 1: Identify when human insight is needed

   Manual testing is best when tests require human judgment, such as usability or visual checks.

2. Step 2: Compare options to manual testing needs

   When tests require human judgment and intuition matches the need for human intuition, unlike options A, B, and C which favor automation.

3. Final Answer:

   When tests require human judgment and intuition -> Option A

4. Quick Check:

   Human insight = Manual testing [OK]

Hint: Manual testing fits tasks needing human judgment [OK]

Common Mistakes:

• Choosing manual for repetitive stable tests
• Confusing automation needs with manual testing
• Ignoring the role of human intuition

3. Consider this scenario: A test suite runs 100 tests daily. 80 tests are stable and repetitive, 20 are new features needing exploration. Which approach is best?

medium

A. Automate all 100 tests

B. Manually test all 100 tests

C. Automate the 80 stable tests and manually test the 20 new features

D. Manually test the 80 stable tests and automate the 20 new features

Solution

1. Step 1: Analyze test types and their nature

   80 tests are stable and repetitive, suitable for automation; 20 are new and exploratory, better for manual testing.

2. Step 2: Match testing approach to test types

   Automate stable tests to save time; manually test new features for human insight.

3. Final Answer:

   Automate the 80 stable tests and manually test the 20 new features -> Option C

4. Quick Check:

   Stable = automate, new = manual [OK]

Hint: Automate stable tests, manual for new features [OK]

Common Mistakes:

• Automating new exploratory tests that change often
• Manually testing all tests wastes time
• Ignoring test stability when choosing method

4. A tester wrote automated scripts for a feature still under frequent change. What is the main problem with this approach?

medium

A. Scripts will require constant updates, wasting effort

B. Automated tests run faster

C. Manual testing is slower

D. Automation tools cannot test new features

Solution

1. Step 1: Understand impact of frequent changes on automation

   When features change often, automated scripts break and need frequent fixes.

2. Step 2: Identify the problem caused by frequent script updates

   Constant updates waste time and reduce automation benefits.

3. Final Answer:

   Scripts will require constant updates, wasting effort -> Option A

4. Quick Check:

   Frequent changes = costly automation maintenance [OK]

Hint: Avoid automating unstable features to save maintenance [OK]

Common Mistakes:

• Thinking automation tools can't test new features
• Confusing speed with suitability
• Ignoring maintenance cost of automation

5. You manage testing for a complex app with frequent UI changes and critical backend logic. How should you balance automation and manual testing?

hard

A. Automate UI tests and manually test backend logic

B. Automate both UI and backend tests fully

C. Manually test both UI and backend tests fully

D. Manually test UI and automate backend logic tests

Solution

1. Step 1: Analyze UI and backend characteristics

   UI changes frequently, making automation brittle; backend logic is stable and critical, suitable for automation.

2. Step 2: Choose testing method per component

   Manual testing fits UI due to frequent changes and human insight; automate backend logic for reliability and speed.

3. Final Answer:

   Manually test UI and automate backend logic tests -> Option D

4. Quick Check:

   UI frequent changes = manual, backend stable = automate [OK]

Hint: Automate stable backend; manual test changing UI [OK]

Common Mistakes:

• Automating frequently changing UI causing flaky tests
• Manually testing stable backend wasting time
• Ignoring test stability and criticality