Test prioritization in Testing Fundamentals - Framework Patterns

Framework Mode - Test prioritization

Folder Structure for Test Prioritization Framework

  test-prioritization-project/
  ├── tests/
  │   ├── critical/
  │   │   ├── test_login.py
  │   │   └── test_payment.py
  │   ├── high/
  │   │   ├── test_user_profile.py
  │   │   └── test_search.py
  │   ├── medium/
  │   │   ├── test_notifications.py
  │   │   └── test_settings.py
  │   └── low/
  │       ├── test_help.py
  │       └── test_about_page.py
  ├── utils/
  │   ├── prioritization.py
  │   └── helpers.py
  ├── config/
  │   ├── environments.yaml
  │   └── test_priorities.yaml
  ├── reports/
  ├── conftest.py
  └── pytest.ini
  

Test Framework Layers for Test Prioritization

• Test Cases Layer: Organized by priority folders (critical, high, medium, low) to run tests based on importance.
• Prioritization Utilities Layer: Contains logic to assign and manage test priorities, e.g., prioritization.py with functions to select tests to run first.
• Configuration Layer: Holds environment settings and priority rules in YAML files for easy updates without code changes.
• Test Runner Layer: Uses Pytest with configuration to run tests selectively by priority.
• Reporting Layer: Collects and presents test results, highlighting priority-based execution outcomes.

Configuration Patterns for Test Prioritization

Use YAML files to define test priorities and environment settings. Example test_priorities.yaml:

  priorities:
    critical:
      - test_login.py
      - test_payment.py
    high:
      - test_user_profile.py
      - test_search.py
    medium:
      - test_notifications.py
      - test_settings.py
    low:
      - test_help.py
      - test_about_page.py
  

Use pytest.ini to configure markers for priorities:

  [pytest]
  markers =
      critical: mark test as critical priority
      high: mark test as high priority
      medium: mark test as medium priority
      low: mark test as low priority
  

Use conftest.py to implement hooks that select tests based on priority markers or config files.

Test Reporting and CI/CD Integration

• Generate reports that show which priority tests passed or failed.
• Use tools like Allure or HTML reports to visualize priority-based test results.
• Integrate with CI/CD pipelines to run critical and high priority tests on every commit, medium and low on nightly builds.
• Configure notifications to alert immediately if critical tests fail.

Best Practices for Test Prioritization Framework

1. Clear Priority Definitions: Define test priorities clearly and keep them updated as the application evolves.
2. Separate Tests by Priority: Organize tests in folders or use markers to easily select tests by priority.
3. Automate Priority Selection: Use configuration files and hooks to automate which tests run based on priority.
4. Run Critical Tests Frequently: Always run critical tests on every code change to catch major issues early.
5. Balance Test Coverage and Speed: Prioritize tests to get fast feedback without sacrificing important coverage.

Self-Check Question

Where in this folder structure would you add a new test case for a critical login feature?

Key Result

Organize and run tests based on their priority to get fast, focused feedback on critical features.