Performance testing basics in Testing Fundamentals - Framework Patterns

Framework Mode - Performance testing basics

Folder Structure for Performance Testing Project

performance-testing-project/
├── scripts/               # Load and stress test scripts
│   ├── login_test.jmx    # Example JMeter script
│   ├── api_load_test.js  # Example k6 script
│   └── user_scenario.js  # Custom user flow scripts
├── results/               # Test run results and reports
│   ├── 2024-06-01_report.html
│   └── 2024-06-01_metrics.json
├── config/                # Configuration files for environments and test parameters
│   ├── environments.yaml
│   └── test_settings.json
├── utils/                 # Helper scripts for data generation, parsing results
│   └── data_generator.py
├── ci/                    # CI/CD pipeline scripts for performance tests
│   └── run_performance_tests.yml
└── README.md              # Project overview and instructions
  

Performance Test Framework Layers

• Test Scripts Layer: Contains the actual performance test scripts that simulate user actions or API calls. Examples: JMeter .jmx files, k6 JavaScript scripts.
• Configuration Layer: Holds environment details (URLs, credentials), test parameters (number of users, duration), and thresholds for pass/fail criteria.
• Utility Layer: Helper code for preparing test data, parsing test results, or generating reports.
• Results Layer: Stores raw and processed test results, logs, and reports for analysis.
• CI/CD Integration Layer: Scripts and configuration to run performance tests automatically in pipelines and publish results.

Configuration Patterns in Performance Testing

• Environment Config: Use YAML or JSON files to define URLs, ports, and credentials for different environments (dev, staging, production).
• Test Parameters: Define user load, ramp-up time, test duration, and thresholds in separate config files to easily adjust without changing scripts.
• Data Management: Use external CSV or JSON files for test data inputs to simulate realistic user behavior.
• Parameterization: Scripts should read config values dynamically to support multiple environments and test scenarios.

Test Reporting and CI/CD Integration

• Reporting: Generate human-readable HTML or PDF reports summarizing response times, throughput, errors, and resource usage.
• Metrics: Collect detailed metrics like average response time, 95th percentile, error rates, and system resource consumption.
• Alerts: Set thresholds to mark tests as pass or fail based on performance goals.
• CI/CD Integration: Automate performance tests in pipelines using scripts (e.g., GitHub Actions, Jenkins). Publish reports as pipeline artifacts or dashboards.

Best Practices for Performance Testing Frameworks

• Keep Tests Maintainable: Separate test logic from configuration to easily update load profiles or environments.
• Use Realistic Scenarios: Simulate real user behavior and data to get meaningful results.
• Automate Reporting: Automatically generate and archive reports for easy comparison over time.
• Integrate Early: Run performance tests regularly in CI/CD to catch issues early.
• Monitor System Resources: Combine performance tests with system monitoring to understand bottlenecks.

Self Check Question

Where in this folder structure would you add a new test script to simulate a user login scenario?

Key Result

Organize performance tests with clear layers: scripts, config, utilities, results, and CI/CD integration for maintainability and automation.

Practice

1. What is the main goal of performance testing in software development?

easy

A. To find spelling mistakes in the user interface

B. To check how fast and stable the software works under heavy use

C. To test if the software installs correctly

D. To check if the software looks good on different screens

Solution

1. Step 1: Understand the purpose of performance testing

   Performance testing measures speed and stability when many users use the software at once.

2. Step 2: Compare options with this purpose

   Only To check how fast and stable the software works under heavy use talks about speed and stability under heavy use, matching the goal.

3. Final Answer:

   To check how fast and stable the software works under heavy use -> Option B

4. Quick Check:

   Performance testing = speed and stability check [OK]

Hint: Performance testing checks speed and stability under load [OK]

Common Mistakes:

• Confusing performance testing with UI or installation testing
• Thinking performance testing checks visual design
• Mixing performance testing with functional testing

2. Which of the following is a common tool used for performance testing?

easy

A. Photoshop

B. WordPress

C. Excel

D. JMeter

Solution

1. Step 1: Identify tools related to performance testing

   JMeter is a popular open-source tool designed for performance testing.

2. Step 2: Eliminate unrelated tools

   Photoshop is for images, Excel for spreadsheets, WordPress for websites, none for performance testing.

3. Final Answer:

   JMeter -> Option D

4. Quick Check:

   Performance tool = JMeter [OK]

Hint: Pick tools known for load testing like JMeter [OK]

Common Mistakes:

• Choosing unrelated software like Photoshop or Excel
• Confusing content management systems with testing tools
• Not knowing popular performance testing tools

3. Consider this scenario: A performance test runs 100 users simultaneously on a website. The average response time is 2 seconds, but 10 users experience 10 seconds delay. What does this indicate?

medium

A. The test is invalid because average time is low

B. The website handles load well with no issues

C. There is a performance bottleneck affecting some users

D. All users have the same experience

Solution

1. Step 1: Analyze the response times

   Most users get 2 seconds, but some have 10 seconds delay, showing inconsistency.

2. Step 2: Understand what inconsistent delays mean

   Long delays for some users suggest a bottleneck or problem under load.

3. Final Answer:

   There is a performance bottleneck affecting some users -> Option C

4. Quick Check:

   High delay for some = bottleneck [OK]

Hint: Look for slow responses in some users to spot bottlenecks [OK]

Common Mistakes:

• Ignoring slow user delays and trusting only average
• Assuming test is invalid without checking details
• Thinking all users have same experience

4. You wrote a performance test script but it always stops with an error after 5 users start. What is the most likely cause?

medium

A. The test script has a limit on concurrent users set too low

B. The website is too fast and closes the test

C. The test tool does not support more than 5 users ever

D. The internet connection is too slow

Solution

1. Step 1: Understand the error context

   The test stops after 5 users start, suggesting a limit reached in the script or tool settings.

2. Step 2: Evaluate options for cause

   The test script has a limit on concurrent users set too low explains a low user limit in the script, which matches the problem. Other options are unlikely causes.

3. Final Answer:

   The test script has a limit on concurrent users set too low -> Option A

4. Quick Check:

   Test stops early = user limit too low [OK]

Hint: Check user limits in script when tests stop early [OK]

Common Mistakes:

• Blaming website speed or internet without evidence
• Assuming tool cannot handle users without checking settings
• Ignoring script configuration for concurrency

5. You want to test a web app's performance under increasing load. Which approach best simulates real user behavior during the test?

hard

A. Gradually increase the number of virtual users over time while running typical user actions

B. Run 1000 users all clicking the same button at the exact same time once

C. Run a single user repeatedly refreshing the homepage for hours

D. Test only the database without simulating user actions

Solution

1. Step 1: Understand realistic load testing

   Real users increase gradually and perform varied actions, not all at once.

2. Step 2: Compare options to realistic behavior

   Gradually increase the number of virtual users over time while running typical user actions simulates gradual load increase with typical actions, matching real use best.

3. Final Answer:

   Gradually increase the number of virtual users over time while running typical user actions -> Option A

4. Quick Check:

   Realistic load = gradual increase + varied actions [OK]

Hint: Simulate real users by increasing load gradually [OK]

Common Mistakes:

• Testing all users clicking simultaneously unrealistically
• Using only one user repeatedly
• Ignoring user actions and testing only backend