Automation ROI calculation in Testing Fundamentals - Build an Automation Script

Calculate Automation ROI for a Test Suite

Preconditions (5)

Step 1: Collect manual test execution time per test case in hours

Step 2: Collect automated test execution time per test case in hours

Step 3: Collect manual test maintenance time per test case in hours

Step 4: Collect automated test maintenance time per test case in hours

Step 5: Collect number of test executions planned

Step 6: Calculate total manual time = (manual execution time + manual maintenance time) * number of executions

Step 7: Calculate total automated time = (automated execution time + automated maintenance time) * number of executions

Step 8: Calculate time saved = total manual time - total automated time

Step 9: Calculate ROI percentage = (time saved / total automated time) * 100

Step 10: Verify ROI percentage is greater than 0

✅ Expected Result: The ROI percentage is calculated correctly and is greater than 0, indicating automation saves time.

Automation Requirements - Python unittest

Assertions Needed:

Assert ROI percentage is a float

Assert ROI percentage is greater than 0

Best Practices:

Use functions to separate calculation logic

Use clear variable names

Use unittest assertions for validation

Handle division by zero if automated time is zero

Automated Solution

Testing Fundamentals

import unittest


def calculate_roi(manual_exec, manual_maint, auto_exec, auto_maint, executions):
    total_manual = (manual_exec + manual_maint) * executions
    total_auto = (auto_exec + auto_maint) * executions
    if total_auto == 0:
        raise ValueError("Automated total time cannot be zero")
    time_saved = total_manual - total_auto
    roi = (time_saved / total_auto) * 100
    return roi


class TestAutomationROICalculation(unittest.TestCase):
    def test_roi_calculation(self):
        manual_exec = 2.0  # hours
        manual_maint = 0.5  # hours
        auto_exec = 0.5  # hours
        auto_maint = 0.2  # hours
        executions = 10

        roi = calculate_roi(manual_exec, manual_maint, auto_exec, auto_maint, executions)

        self.assertIsInstance(roi, float, "ROI should be a float")
        self.assertGreater(roi, 0, "ROI should be greater than 0")

    def test_zero_automated_time(self):
        with self.assertRaises(ValueError):
            calculate_roi(1, 1, 0, 0, 5)


if __name__ == '__main__':
    unittest.main()

The calculate_roi function computes the return on investment percentage by first calculating total manual and automated times, then computing the time saved and ROI percentage. It raises an error if automated total time is zero to avoid division by zero.

The TestAutomationROICalculation class uses Python's unittest framework to test the ROI calculation. It checks that the ROI is a float and greater than zero, matching the expected result. It also tests that the function raises an error when automated time is zero.

This structure separates calculation logic from tests, uses clear variable names, and applies proper assertions for validation.

Common Mistakes - 4 Pitfalls

Not handling division by zero when automated time is zero

{'mistake': "Using unclear variable names like 'a', 'b', 'c'", 'why_bad': 'Makes code hard to read and understand.', 'correct_approach': 'Use descriptive names like manual_exec, auto_maint, executions.'}

Not using assertions to check ROI type and value

Mixing calculation logic inside test methods

Bonus Challenge

Now add data-driven testing with 3 different sets of inputs for manual and automated times and executions.

Show Hint

Practice

1. What does Automation ROI primarily measure in software testing?

easy

A. The cost savings from using automated tests compared to manual tests

B. The number of test cases automated

C. The speed of writing test scripts

D. The number of bugs found by automation

Solution

1. Step 1: Understand ROI meaning

   ROI (Return on Investment) measures how much money or value you get back from what you spend.

2. Step 2: Apply ROI to automation context

   In testing, Automation ROI compares the cost of automating tests to the money saved by running tests automatically instead of manually.

3. Final Answer:

   The cost savings from using automated tests compared to manual tests -> Option A

4. Quick Check:

   Automation ROI = Cost savings [OK]

Hint: ROI means money saved versus money spent [OK]

Common Mistakes:

• Confusing ROI with number of tests automated
• Thinking ROI measures speed only
• Believing ROI counts bugs found

2. Which formula correctly calculates Automation ROI?

easy

A. (Cost of Automation + Cost of Manual Testing) / Cost of Automation

B. (Cost of Automation - Cost of Manual Testing) / Cost of Automation

C. (Cost of Manual Testing - Cost of Automation) / Cost of Automation

D. (Cost of Manual Testing + Cost of Automation) / Cost of Manual Testing

Solution

1. Step 1: Recall ROI formula basics

   ROI = (Gain from investment - Cost of investment) / Cost of investment.

2. Step 2: Apply to automation costs

   Gain = Cost saved by automation = Cost of Manual Testing - Cost of Automation. So ROI = (Manual - Automation) / Automation.

3. Final Answer:

   (Cost of Manual Testing - Cost of Automation) / Cost of Automation -> Option C

4. Quick Check:

   ROI = (Manual - Automation) / Automation [OK]

Hint: ROI = (Manual cost - Automation cost) ÷ Automation cost [OK]

Common Mistakes:

• Reversing subtraction order
• Adding costs instead of subtracting
• Dividing by wrong cost

3. Given: Manual testing cost = $10,000, Automation cost = $4,000. What is the Automation ROI?

medium

A. 1.5

B. 0.6

C. 1.0

D. 0.4

Solution

1. Step 1: Identify values and formula

   Manual cost = 10000, Automation cost = 4000. ROI = (Manual - Automation) / Automation.

2. Step 2: Calculate ROI

   ROI = (10000 - 4000) / 4000 = 6000 / 4000 = 1.5.

3. Final Answer:

   1.5 -> Option A

4. Quick Check:

   ROI = (10000-4000)/4000 = 1.5 [OK]

Hint: Subtract then divide by automation cost [OK]

Common Mistakes:

• Dividing by manual cost instead of automation
• Adding costs instead of subtracting
• Mixing up manual and automation values

4. A tester calculates Automation ROI as (Automation Cost - Manual Cost) / Automation Cost. What is wrong with this formula?

medium

A. It divides by manual cost instead of automation cost

B. It subtracts in the wrong order, giving negative ROI when automation saves money

C. It adds costs instead of subtracting

D. It does not include test execution time

Solution

1. Step 1: Compare given formula to correct formula

   Correct ROI = (Manual Cost - Automation Cost) / Automation Cost. Given formula reverses subtraction.

2. Step 2: Understand impact of reversed subtraction

   If automation costs less, reversing subtraction yields negative ROI, wrongly suggesting loss.

3. Final Answer:

   It subtracts in the wrong order, giving negative ROI when automation saves money -> Option B

4. Quick Check:

   Subtraction order matters for correct ROI sign [OK]

Hint: Subtract manual minus automation, not reverse [OK]

Common Mistakes:

• Reversing subtraction order
• Dividing by wrong cost
• Ignoring subtraction sign impact

5. A company spends $15,000 on manual testing yearly. Automating tests costs $12,000 upfront but saves $5,000 yearly in manual effort. After 3 years, what is the total Automation ROI?

hard

A. 1.25

B. 0.5

C. 0.75

D. 0.25

Solution

1. Step 1: Calculate total savings over 3 years

   Savings = $5,000/year x 3 years = $15,000.

2. Step 2: Identify automation investment

   Cost of automation = $12,000 upfront.

3. Step 3: Calculate ROI

   ROI = (Savings - Automation cost) / Automation cost = (15,000 - 12,000) / 12,000 = 3,000 / 12,000 = 0.25.

4. Final Answer:

   0.25 -> Option D

5. Quick Check:

   ROI = (15000 - 12000) / 12000 = 0.25 [OK]

Hint: Calculate total costs and savings over years before ROI [OK]

Common Mistakes:

• Ignoring yearly savings in calculation
• Not multiplying costs by years
• Mixing upfront and recurring costs