Stress testing concepts in Testing Fundamentals - Framework Patterns

Framework Mode - Stress testing concepts

Folder Structure for Stress Testing Project

stress-testing-project/
├── tests/
│   ├── stress/
│   │   ├── cpu_stress_test.py
│   │   ├── memory_stress_test.py
│   │   ├── io_stress_test.py
│   │   └── network_stress_test.py
│   └── __init__.py
├── utils/
│   ├── resource_monitor.py
│   ├── load_generator.py
│   └── __init__.py
├── config/
│   ├── environments.yaml
│   ├── stress_test_config.yaml
│   └── __init__.py
├── reports/
│   └── stress_test_report.html
├── conftest.py
└── README.md
  

Test Framework Layers for Stress Testing

• Test Scripts Layer: Contains stress test scripts targeting CPU, memory, disk I/O, and network stress scenarios.
• Utilities Layer: Helper modules for generating load, monitoring system resources, and collecting metrics.
• Configuration Layer: Holds environment settings, thresholds, and test parameters for different test runs.
• Reporting Layer: Generates detailed reports on system behavior under stress, including pass/fail and resource usage graphs.
• Test Runner & Fixtures: Setup and teardown logic to prepare the environment and clean up after tests.

Configuration Patterns for Stress Testing

• Environment Config: Use YAML files to define different environments (dev, staging, production) with resource limits and access credentials.
• Test Parameters: Define stress levels (e.g., CPU load %, memory size) in config files to easily adjust test intensity without code changes.
• Dynamic Overrides: Allow command-line arguments or environment variables to override config values for flexible test runs.
• Credentials Management: Store sensitive data securely and load them at runtime, avoiding hardcoding in test scripts.

Test Reporting and CI/CD Integration

• Detailed Reports: Generate HTML or JSON reports showing resource usage graphs, error logs, and pass/fail status.
• Alerts: Integrate with notification tools (email, Slack) to alert on failures or threshold breaches.
• CI/CD Integration: Run stress tests as part of nightly builds or pre-release pipelines to catch performance issues early.
• Historical Data: Store reports to track performance trends over time and identify regressions.

Best Practices for Stress Testing Frameworks

• Isolate Tests: Run stress tests in isolated environments to avoid affecting production systems.
• Use Realistic Load: Simulate real user behavior or system load patterns for meaningful results.
• Monitor Continuously: Collect system metrics continuously during tests to detect bottlenecks early.
• Automate Cleanup: Ensure tests clean up resources to prevent side effects on subsequent tests.
• Parameterize Tests: Make stress levels configurable to test different scenarios without code changes.

Self-Check Question

Where in this folder structure would you add a new stress test script for network bandwidth?

Key Result

Organize stress testing with clear layers: test scripts, utilities, config, reporting, and integrate with CI/CD for automated performance checks.

Practice

1. What is the main goal of stress testing in software testing?

easy

A. To verify the system's installation process

B. To check if the system meets all functional requirements

C. To test the system's user interface for usability

D. To push the system beyond its normal limits to find breaking points

Solution

1. Step 1: Understand the purpose of stress testing

   Stress testing is designed to push a system beyond normal operational capacity to see how it behaves under extreme conditions.

2. Step 2: Compare options with stress testing goal

   Only To push the system beyond its normal limits to find breaking points describes pushing the system beyond normal limits to find breaking points, which matches stress testing's main goal.

3. Final Answer:

   To push the system beyond its normal limits to find breaking points -> Option D

4. Quick Check:

   Stress testing = pushing limits [OK]

Hint: Stress testing means pushing beyond normal use [OK]

Common Mistakes:

• Confusing stress testing with functional testing
• Thinking stress testing checks UI or installation
• Assuming stress testing is about normal load only

2. Which of the following is the correct way to describe a stress test scenario?

easy

A. Testing the system with a load that exceeds its maximum capacity

B. Testing the system with normal expected user load

C. Testing only the system's database queries

D. Testing the system's response to a single user action

Solution

1. Step 1: Define stress test scenario

   A stress test scenario involves applying a load that is higher than the system's maximum expected capacity to observe failure points.

2. Step 2: Match options to definition

   Testing the system with a load that exceeds its maximum capacity correctly states testing with load exceeding maximum capacity, which fits stress testing. Other options describe normal load or unrelated tests.

3. Final Answer:

   Testing the system with a load that exceeds its maximum capacity -> Option A

4. Quick Check:

   Stress test = load beyond max [OK]

Hint: Stress test means load beyond max capacity [OK]

Common Mistakes:

• Choosing normal load as stress test
• Confusing stress test with functional or unit tests
• Ignoring the 'exceeds capacity' part

3. Consider this pseudo-code for a stress test:

for users in range(1000, 1100):
    simulate_login(users)
    simulate_data_request(users)
    check_response_time(users)

What is the main purpose of increasing users from 1000 to 1099 in this test?

medium

A. To check if the system crashes at 1000 users

B. To verify system functionality for exactly 1000 users

C. To test system behavior under gradually increasing load

D. To test only the login feature repeatedly

Solution

1. Step 1: Analyze the loop range and actions

   The loop increases user count from 1000 to 1099, simulating login and data requests for each user count, then checking response time.

2. Step 2: Understand the test goal

   This gradual increase tests how the system handles increasing load, which is typical in stress testing to find breaking points or performance degradation.

3. Final Answer:

   To test system behavior under gradually increasing load -> Option C

4. Quick Check:

   Increasing users = increasing load [OK]

Hint: Increasing users means testing load growth [OK]

Common Mistakes:

• Thinking test is only for exactly 1000 users
• Assuming test checks only login feature
• Ignoring the gradual increase aspect

4. A tester wrote this stress test code snippet:

for i in range(1000, 1100):
    simulate_login(i)
    simulate_data_request(i)
    check_response_time()

What is the main issue in this code?

medium

A. The check_response_time function is missing the user parameter

B. The loop range is incorrect for stress testing

C. The simulate_login function should be outside the loop

D. There is no issue; the code is correct

Solution

1. Step 1: Review function calls inside the loop

   The loop calls simulate_login(i) and simulate_data_request(i) with the user parameter i. However, check_response_time() is called without any parameter.

2. Step 2: Identify inconsistency

   Since response time likely depends on the user or request, missing the user parameter in check_response_time() is an error causing incorrect or failed measurement.

3. Final Answer:

   The check_response_time function is missing the user parameter -> Option A

4. Quick Check:

   Missing parameter in function call = bug [OK]

Hint: Check all function calls have needed parameters [OK]

Common Mistakes:

• Ignoring missing parameters in function calls
• Thinking loop range is wrong for stress testing
• Assuming functions should be outside the loop

5. You want to design a stress test for a web application that must handle up to 5000 users simultaneously. Which approach best fits stress testing principles?

hard

A. Test the system with 3000 users and assume it works for 5000

B. Gradually increase users from 1000 to 6000 while monitoring system response and failures

C. Run functional tests for 100 users repeatedly

D. Test only 5000 users once and check if the system crashes

Solution

1. Step 1: Understand stress testing for capacity limits

   Stress testing involves pushing the system beyond expected limits to find breaking points and observe behavior under extreme load.

2. Step 2: Evaluate options for best stress test design

   Gradually increase users from 1000 to 6000 while monitoring system response and failures gradually increases users beyond 5000 to 6000, monitoring response and failures, which aligns with stress testing goals. Other options either test only once, test fewer users, or do unrelated functional tests.

3. Final Answer:

   Gradually increase users from 1000 to 6000 while monitoring system response and failures -> Option B

4. Quick Check:

   Gradual increase beyond max = proper stress test [OK]

Hint: Increase load beyond max users gradually [OK]

Common Mistakes:

• Testing only once at max users
• Confusing stress test with functional test
• Assuming fewer users prove max capacity