When to automate vs manual test in Testing Fundamentals - Test Execution Compared

Test Overview

This test checks if the tester correctly decides when to use automated testing versus manual testing based on test case characteristics.

Test Code - PyTest

Testing Fundamentals

def decide_test_method(test_case):
    """Decide whether to automate or do manual testing based on test case properties."""
    if test_case['is_repetitive'] and test_case['stable_requirements'] and test_case['high_risk_area']:
        return 'Automate'
    elif test_case['requires_exploratory'] or test_case['one_time_test']:
        return 'Manual'
    else:
        return 'Manual'

# Example test cases
repetitive_test = {'is_repetitive': True, 'stable_requirements': True, 'high_risk_area': True, 'requires_exploratory': False, 'one_time_test': False}
exploratory_test = {'is_repetitive': False, 'stable_requirements': False, 'high_risk_area': False, 'requires_exploratory': True, 'one_time_test': False}
one_time_test = {'is_repetitive': False, 'stable_requirements': True, 'high_risk_area': False, 'requires_exploratory': False, 'one_time_test': True}

assert decide_test_method(repetitive_test) == 'Automate'
assert decide_test_method(exploratory_test) == 'Manual'
assert decide_test_method(one_time_test) == 'Manual'

Execution Trace - 5 Steps

Step	Action	System State	Assertion	Result
1	Test starts	Test function decide_test_method is ready to run with example test cases	—	PASS
2	Call decide_test_method with repetitive_test case	Input: {'is_repetitive': True, 'stable_requirements': True, 'high_risk_area': True, 'requires_exploratory': False, 'one_time_test': False}	Return value should be 'Automate'	PASS
3	Call decide_test_method with exploratory_test case	Input: {'is_repetitive': False, 'stable_requirements': False, 'high_risk_area': False, 'requires_exploratory': True, 'one_time_test': False}	Return value should be 'Manual'	PASS
4	Call decide_test_method with one_time_test case	Input: {'is_repetitive': False, 'stable_requirements': True, 'high_risk_area': False, 'requires_exploratory': False, 'one_time_test': True}	Return value should be 'Manual'	PASS
5	All assertions checked	All test cases returned expected results	All assert statements passed	PASS

Failure Scenario

Failing Condition: The decide_test_method returns wrong test method for any test case

Execution Trace Quiz - 3 Questions

Which test case should be automated according to the test?

A A one-time exploratory test

B A test requiring manual exploration

C A repetitive test with stable requirements and high risk

D A test with unstable requirements

Key Result

Automate tests that are repetitive, stable, and high risk to save time and reduce errors. Use manual testing for exploratory or one-time tests where human insight is needed.

Practice

1. Which type of testing is best suited for repetitive tasks that rarely change?

easy

A. Manual testing

B. Automated testing

C. Exploratory testing

D. Ad-hoc testing

Solution

1. Step 1: Understand the nature of repetitive tasks

   Repetitive tasks are those that happen often and have stable expected results.

2. Step 2: Match testing type to task nature

   Automated testing is ideal for repetitive and stable tasks because it saves time and reduces human error.

3. Final Answer:

   Automated testing -> Option B

4. Quick Check:

   Repetitive tasks = Automated testing [OK]

Hint: Automate stable, repetitive tests to save time [OK]

Common Mistakes:

• Choosing manual testing for repetitive tasks wastes time
• Confusing exploratory testing with repetitive testing
• Thinking ad-hoc testing fits repetitive tasks

2. Which of the following is the correct reason to choose manual testing over automation?

easy

A. When tests require human judgment and intuition

B. When tests are simple and can be scripted easily

C. When tests need to run overnight without interruption

D. When tests are repetitive and stable

Solution

1. Step 1: Identify when human insight is needed

   Manual testing is best when tests require human judgment, such as usability or visual checks.

2. Step 2: Compare options to manual testing needs

   When tests require human judgment and intuition matches the need for human intuition, unlike options A, B, and C which favor automation.

3. Final Answer:

   When tests require human judgment and intuition -> Option A

4. Quick Check:

   Human insight = Manual testing [OK]

Hint: Manual testing fits tasks needing human judgment [OK]

Common Mistakes:

• Choosing manual for repetitive stable tests
• Confusing automation needs with manual testing
• Ignoring the role of human intuition

3. Consider this scenario: A test suite runs 100 tests daily. 80 tests are stable and repetitive, 20 are new features needing exploration. Which approach is best?

medium

A. Automate all 100 tests

B. Manually test all 100 tests

C. Automate the 80 stable tests and manually test the 20 new features

D. Manually test the 80 stable tests and automate the 20 new features

Solution

1. Step 1: Analyze test types and their nature

   80 tests are stable and repetitive, suitable for automation; 20 are new and exploratory, better for manual testing.

2. Step 2: Match testing approach to test types

   Automate stable tests to save time; manually test new features for human insight.

3. Final Answer:

   Automate the 80 stable tests and manually test the 20 new features -> Option C

4. Quick Check:

   Stable = automate, new = manual [OK]

Hint: Automate stable tests, manual for new features [OK]

Common Mistakes:

• Automating new exploratory tests that change often
• Manually testing all tests wastes time
• Ignoring test stability when choosing method

4. A tester wrote automated scripts for a feature still under frequent change. What is the main problem with this approach?

medium

A. Scripts will require constant updates, wasting effort

B. Automated tests run faster

C. Manual testing is slower

D. Automation tools cannot test new features

Solution

1. Step 1: Understand impact of frequent changes on automation

   When features change often, automated scripts break and need frequent fixes.

2. Step 2: Identify the problem caused by frequent script updates

   Constant updates waste time and reduce automation benefits.

3. Final Answer:

   Scripts will require constant updates, wasting effort -> Option A

4. Quick Check:

   Frequent changes = costly automation maintenance [OK]

Hint: Avoid automating unstable features to save maintenance [OK]

Common Mistakes:

• Thinking automation tools can't test new features
• Confusing speed with suitability
• Ignoring maintenance cost of automation

5. You manage testing for a complex app with frequent UI changes and critical backend logic. How should you balance automation and manual testing?

hard

A. Automate UI tests and manually test backend logic

B. Automate both UI and backend tests fully

C. Manually test both UI and backend tests fully

D. Manually test UI and automate backend logic tests

Solution

1. Step 1: Analyze UI and backend characteristics

   UI changes frequently, making automation brittle; backend logic is stable and critical, suitable for automation.

2. Step 2: Choose testing method per component

   Manual testing fits UI due to frequent changes and human insight; automate backend logic for reliability and speed.

3. Final Answer:

   Manually test UI and automate backend logic tests -> Option D

4. Quick Check:

   UI frequent changes = manual, backend stable = automate [OK]

Hint: Automate stable backend; manual test changing UI [OK]

Common Mistakes:

• Automating frequently changing UI causing flaky tests
• Manually testing stable backend wasting time
• Ignoring test stability and criticality