Why non-functional quality affects user experience in Testing Fundamentals - Framework Benefits

Framework Mode - Why non-functional quality affects user experience

Folder Structure Example

non-functional-quality-testing/
├── tests/
│   ├── performance/
│   │   └── load_test.py
│   ├── security/
│   │   └── auth_test.py
│   ├── usability/
│   │   └── accessibility_test.py
│   └── reliability/
│       └── stability_test.py
├── utils/
│   ├── logger.py
│   ├── metrics_collector.py
│   └── environment.py
├── config/
│   ├── environments.yaml
│   └── thresholds.yaml
├── reports/
│   └── latest_report.html
└── conftest.py
  

Test Framework Layers

• Test Cases Layer: Contains tests for non-functional aspects like performance, security, usability, and reliability.
• Utilities Layer: Helper modules for logging, metrics collection, and environment setup.
• Configuration Layer: Holds environment settings and quality thresholds to compare test results.
• Reporting Layer: Generates readable reports showing if non-functional goals are met.

Configuration Patterns

Use environments.yaml to define URLs, credentials, and environment-specific settings.

Use thresholds.yaml to set acceptable limits for response times, error rates, and accessibility scores.

Load configurations dynamically in tests to run the same tests across different environments and quality goals.

Test Reporting and CI/CD Integration

Generate HTML or JSON reports summarizing non-functional test results.

Integrate with CI/CD pipelines to run non-functional tests automatically on new builds.

Use reports to quickly see if performance or security goals are met before releasing.

Framework Design Principles

• Separate Non-Functional Tests: Keep performance, security, usability tests in their own folders for clarity.
• Use Clear Thresholds: Define measurable goals to decide pass/fail for non-functional tests.
• Automate Reporting: Make reports easy to read and integrate with build tools for fast feedback.
• Reuse Utilities: Share logging and metrics code to keep tests simple and consistent.
• Test Realistic Scenarios: Simulate real user conditions to see true impact on user experience.

Self-Check Question

Where would you add a new test to check the website's load time under heavy user traffic?

Key Result

Organize non-functional tests separately with clear thresholds and automated reporting to ensure quality impacts user experience positively.

Practice

1. Which of the following is an example of a non-functional quality that affects user experience?

easy

A. The color scheme used in the user interface

B. The number of features available in the software

C. How fast the software responds to user actions

D. The programming language used to build the software

Solution

1. Step 1: Understand non-functional qualities

   Non-functional qualities describe how software performs, like speed or security, not what it does.

2. Step 2: Identify the example related to performance

   Speed of response is a classic non-functional quality affecting user experience.

3. Final Answer:

   How fast the software responds to user actions -> Option C

4. Quick Check:

   Non-functional quality = speed [OK]

Hint: Non-functional = how software works, not what it does [OK]

Common Mistakes:

• Confusing features (functional) with qualities (non-functional)
• Thinking UI colors are always non-functional
• Assuming programming language affects user experience directly

2. Which statement correctly describes a non-functional requirement in software testing?

easy

A. It defines the system's performance and security standards.

B. It specifies the exact steps a user must follow to complete a task.

C. It lists all the features the software must have.

D. It describes the programming language syntax used.

Solution

1. Step 1: Recall what non-functional requirements are

   They describe qualities like performance, security, usability, not specific features or steps.

2. Step 2: Match the correct description

   Performance and security standards fit non-functional requirements perfectly.

3. Final Answer:

   It defines the system's performance and security standards. -> Option A

4. Quick Check:

   Non-functional requirement = performance/security [OK]

Hint: Non-functional = qualities like speed and security [OK]

Common Mistakes:

• Confusing functional steps with non-functional requirements
• Thinking programming language details are requirements
• Mixing features with quality standards

3. Consider this scenario: A website loads slowly and sometimes crashes under heavy traffic. Which non-functional qualities are most affected?

medium

A. Usability and maintainability

B. Performance and reliability

C. Security and accessibility

D. Functionality and compatibility

Solution

1. Step 1: Identify the issues described

   Slow loading relates to performance; crashing relates to reliability.

2. Step 2: Match issues to non-functional qualities

   Performance means speed; reliability means stability under load.

3. Final Answer:

   Performance and reliability -> Option B

4. Quick Check:

   Slow + crash = performance + reliability [OK]

Hint: Slow and crash? Think performance and reliability [OK]

Common Mistakes:

• Confusing usability with performance
• Mixing security with crashing issues
• Assuming functionality covers speed problems

4. A software tester notices that users complain about confusing navigation and frequent security warnings. Which non-functional qualities should the tester focus on improving?

medium

A. Compatibility and functionality

B. Performance and scalability

C. Reliability and maintainability

D. Usability and security

Solution

1. Step 1: Analyze user complaints

   Confusing navigation relates to usability; security warnings relate to security.

2. Step 2: Identify matching non-functional qualities

   Usability means ease of use; security means protection from threats.

3. Final Answer:

   Usability and security -> Option D

4. Quick Check:

   Navigation + warnings = usability + security [OK]

Hint: Confusing UI + warnings = usability and security [OK]

Common Mistakes:

• Focusing on performance instead of usability
• Ignoring security warnings as minor issues
• Mixing functionality with usability

5. A mobile app has excellent features but users report it drains battery quickly and often freezes. How should the development team prioritize non-functional testing to improve user experience?

hard

A. Focus on battery usage and stability tests before adding new features

B. Add more features to distract users from battery issues

C. Ignore battery issues and focus on UI design only

D. Test only security since freezing is unrelated to performance

Solution

1. Step 1: Understand user complaints impact

   Battery drain affects usability and user satisfaction; freezing affects reliability and stability.

2. Step 2: Decide testing priorities

   Improving battery usage and stability directly improves user experience before adding features.

3. Step 3: Eliminate incorrect options

   Adding features or ignoring battery issues worsens experience; freezing relates to performance, not just security.

4. Final Answer:

   Focus on battery usage and stability tests before adding new features -> Option A

5. Quick Check:

   Battery + freeze = test usage + stability first [OK]

Hint: Fix battery and freezing issues before new features [OK]

Common Mistakes:

• Prioritizing new features over fixing core issues
• Ignoring battery drain as unimportant
• Misunderstanding freezing as only a security problem