Why non-functional quality affects user experience in Testing Fundamentals - Framework Benefits

Framework Mode - Why non-functional quality affects user experience

Folder Structure Example

non-functional-quality-testing/
├── tests/
│   ├── performance/
│   │   └── load_test.py
│   ├── security/
│   │   └── auth_test.py
│   ├── usability/
│   │   └── accessibility_test.py
│   └── reliability/
│       └── stability_test.py
├── utils/
│   ├── logger.py
│   ├── metrics_collector.py
│   └── environment.py
├── config/
│   ├── environments.yaml
│   └── thresholds.yaml
├── reports/
│   └── latest_report.html
└── conftest.py
  

Test Framework Layers

• Test Cases Layer: Contains tests for non-functional aspects like performance, security, usability, and reliability.
• Utilities Layer: Helper modules for logging, metrics collection, and environment setup.
• Configuration Layer: Holds environment settings and quality thresholds to compare test results.
• Reporting Layer: Generates readable reports showing if non-functional goals are met.

Configuration Patterns

Use environments.yaml to define URLs, credentials, and environment-specific settings.

Use thresholds.yaml to set acceptable limits for response times, error rates, and accessibility scores.

Load configurations dynamically in tests to run the same tests across different environments and quality goals.

Test Reporting and CI/CD Integration

Generate HTML or JSON reports summarizing non-functional test results.

Integrate with CI/CD pipelines to run non-functional tests automatically on new builds.

Use reports to quickly see if performance or security goals are met before releasing.

Framework Design Principles

• Separate Non-Functional Tests: Keep performance, security, usability tests in their own folders for clarity.
• Use Clear Thresholds: Define measurable goals to decide pass/fail for non-functional tests.
• Automate Reporting: Make reports easy to read and integrate with build tools for fast feedback.
• Reuse Utilities: Share logging and metrics code to keep tests simple and consistent.
• Test Realistic Scenarios: Simulate real user conditions to see true impact on user experience.

Self-Check Question

Where would you add a new test to check the website's load time under heavy user traffic?

Key Result

Organize non-functional tests separately with clear thresholds and automated reporting to ensure quality impacts user experience positively.