Regression testing in Testing Fundamentals - Framework Patterns

Framework Mode - Regression testing

Folder Structure for Regression Testing Framework

regression-testing-framework/
├── tests/
│   ├── regression/
│   │   ├── test_login.py
│   │   ├── test_checkout.py
│   │   └── test_profile_update.py
│   └── __init__.py
├── pages/
│   ├── login_page.py
│   ├── checkout_page.py
│   └── profile_page.py
├── utils/
│   ├── helpers.py
│   └── data_loader.py
├── config/
│   ├── config.yaml
│   └── credentials.yaml
├── reports/
│   └── latest_report.html
├── conftest.py
└── pytest.ini
  

Test Framework Layers

• Test Layer: Contains regression test scripts that verify existing features still work after changes.
• Page Object Layer: Encapsulates page elements and actions for reusability and easy maintenance.
• Utility Layer: Helper functions for data handling, common actions, and test support.
• Configuration Layer: Stores environment settings, credentials, and test parameters.
• Reporting Layer: Generates readable test reports showing pass/fail results.

Configuration Patterns

Use YAML files to store environment URLs, browser types, and user credentials separately.

Example config.yaml:

environment:
  base_url: "https://staging.example.com"
  browser: "chrome"

logging:
  level: "INFO"
  

Use conftest.py to load configs and provide fixtures for tests to access settings dynamically.

Test Reporting and CI/CD Integration

• Use pytest-html plugin to generate HTML reports after test runs.
• Reports show which regression tests passed or failed with details.
• Integrate with CI/CD pipelines (e.g., GitHub Actions, Jenkins) to run regression tests automatically on code changes.
• Fail builds if critical regression tests fail to prevent broken releases.

Best Practices for Regression Testing Framework

1. Maintain Test Independence: Each regression test should run independently to avoid false failures.
2. Use Page Object Model: Keep UI locators and actions separate from test logic for easier updates.
3. Automate Test Data Setup: Prepare consistent test data to ensure reliable regression results.
4. Run Tests Frequently: Schedule regression tests to run on every code commit or nightly builds.
5. Keep Tests Focused: Regression tests should cover critical existing features without testing new functionality.

Self Check

Where in this folder structure would you add a new regression test for the "Forgot Password" feature?

Key Result

Organize regression tests with clear layers: tests, page objects, utilities, config, and reporting for maintainable and reliable automation.

Practice

1.

What is the main purpose of regression testing in data science projects?

easy

A. To check that new changes do not break existing results

B. To create new models from scratch

C. To collect new data for training

D. To visualize data trends

Solution

1. Step 1: Understand regression testing goal

   Regression testing is about verifying that recent changes do not cause errors in existing functionality.

2. Step 2: Match purpose with options

   Only To check that new changes do not break existing results describes this goal correctly, others describe unrelated tasks.

3. Final Answer:

   To check that new changes do not break existing results -> Option A

4. Quick Check:

   Regression testing = check no breakage [OK]

Hint: Regression testing ensures old results still work after changes [OK]

Common Mistakes:

• Confusing regression testing with model building
• Thinking it collects new data
• Assuming it is for data visualization

2.

Which of the following is the correct way to describe regression testing?

Regression testing:

def regression_test(old_output, new_output):
    return old_output == new_output

easy

A. It compares old and new outputs to detect changes

B. It trains a new model using old and new data

C. It visualizes differences between datasets

D. It deletes old test results before running new tests

Solution

1. Step 1: Analyze the function code

   The function returns True if old_output equals new_output, meaning it compares outputs.

2. Step 2: Match function behavior to options

   It compares old and new outputs to detect changes correctly states it compares outputs to detect changes; others describe unrelated actions.

3. Final Answer:

   It compares old and new outputs to detect changes -> Option A

4. Quick Check:

   Function compares outputs = It compares old and new outputs to detect changes [OK]

Hint: Regression test checks if old and new outputs match exactly [OK]

Common Mistakes:

• Thinking regression testing retrains models
• Assuming it creates visualizations
• Believing it deletes old results

3.

Given the following Python code for regression testing outputs, what will be the printed result?

old_results = [10, 20, 30]
new_results = [10, 20, 25]

if old_results == new_results:
    print("Test Passed")
else:
    print("Test Failed")

medium

A. Test Passed

B. SyntaxError

C. Test Failed

D. No output

Solution

1. Step 1: Compare the two lists element-wise

   old_results has [10, 20, 30], new_results has [10, 20, 25]. The last elements differ (30 vs 25).

2. Step 2: Evaluate the if condition

   Since the lists are not equal, the else block runs, printing "Test Failed".

3. Final Answer:

   Test Failed -> Option C

4. Quick Check:

   Lists differ, so output = Test Failed [OK]

Hint: Lists must match exactly for test to pass [OK]

Common Mistakes:

• Assuming partial match passes test
• Confusing list equality with set equality
• Expecting syntax errors from correct code

4.

Identify the error in this regression testing code snippet:

def regression_test(old, new):
    if old = new:
        return True
    else:
        return False

medium

A. Missing return statement

B. Using assignment operator '=' instead of comparison '==' in if condition

C. Function should print results, not return

D. Incorrect function name

Solution

1. Step 1: Check the if condition syntax

   The condition uses '=' which is assignment, not comparison. Python requires '==' for comparison.

2. Step 2: Identify correct syntax for comparison

   Replacing '=' with '==' fixes the syntax error and allows proper comparison.

3. Final Answer:

   Using assignment operator '=' instead of comparison '==' in if condition -> Option B

4. Quick Check:

   Use '==' to compare, not '=' [OK]

Hint: Use '==' for comparison, '=' is assignment [OK]

Common Mistakes:

• Confusing '=' and '==' in conditions
• Thinking return is missing
• Believing function name matters for syntax

5.

You have a data science model that outputs predictions as a list. You want to automate regression testing to check if new code changes break the model's output. Which approach below best applies regression testing?

hard

A. Delete old predictions and start fresh after every change

B. Train a new model with updated data and ignore old predictions

C. Visualize new predictions only to check for differences

D. Save previous predictions and compare them with new predictions after changes

Solution

1. Step 1: Understand regression testing in model outputs

   Regression testing means re-running tests and comparing new outputs to old outputs to detect unintended changes.

2. Step 2: Evaluate options for best fit

   Save previous predictions and compare them with new predictions after changes saves old predictions and compares them to new ones, matching regression testing. Others ignore old outputs or delete them, which is not regression testing.

3. Final Answer:

   Save previous predictions and compare them with new predictions after changes -> Option D

4. Quick Check:

   Regression testing = compare old and new outputs [OK]

Hint: Keep old outputs to compare with new after changes [OK]

Common Mistakes:

• Ignoring old outputs in testing
• Assuming visualization replaces regression tests
• Deleting old results before testing