Performance metrics collection in Selenium Python - Framework Patterns

Framework Mode - Performance metrics collection

Folder Structure

performance_metrics_project/
├── tests/
│   ├── test_performance.py
│   └── __init__.py
├── pages/
│   ├── base_page.py
│   └── home_page.py
├── utils/
│   ├── performance_helper.py
│   └── __init__.py
├── config/
│   ├── config.yaml
│   └── __init__.py
├── reports/
│   └── performance_report.html
├── conftest.py
└── pytest.ini

Test Framework Layers

• Driver Layer: Selenium WebDriver setup and teardown managed in conftest.py using pytest fixtures.
• Page Objects: Classes in pages/ folder representing web pages, encapsulating element locators and actions.
• Tests: Test scripts in tests/ folder that use page objects and utilities to perform performance metric collection and assertions.
• Utilities: Helper functions in utils/performance_helper.py to extract and process performance metrics from browser logs or JavaScript.
• Configuration: Environment settings, URLs, browser options, and credentials stored in config/config.yaml and loaded at runtime.
• Reports: Generated performance reports saved in reports/ folder, e.g., HTML or JSON format.

Configuration Patterns

• Environment Config: Use config/config.yaml to define URLs, browser types, and timeouts for different environments (dev, staging, prod).
• Browser Options: Configure Selenium WebDriver options (headless, logging preferences) in conftest.py based on config values.
• Credentials: Store sensitive data securely outside the repo or use environment variables; load them in fixtures if needed.
• Dynamic Config Loading: Use pytest command line options or environment variables to select environment and browser at test runtime.

Test Reporting and CI/CD Integration

• Performance Reports: Generate HTML or JSON reports summarizing metrics like page load time, resource timings, and custom timings using utilities.
• Logging: Capture browser console logs and performance logs during test execution for detailed analysis.
• CI/CD Integration: Integrate tests with CI tools (e.g., GitHub Actions, Jenkins) to run performance tests on each build or deployment.
• Alerts: Configure thresholds for performance metrics and fail tests or send notifications if thresholds are exceeded.

Framework Design Principles

1. Separation of Concerns: Keep page objects, test logic, and performance metric extraction separate for clarity and reuse.
2. Use Browser Performance APIs: Leverage browser APIs like Navigation Timing and Resource Timing for accurate metrics.
3. Explicit Waits: Use explicit waits to ensure page elements are loaded before collecting performance data.
4. Configurable Thresholds: Allow performance thresholds to be configurable to adapt to different environments or test goals.
5. Reusable Utilities: Create helper functions to parse and format performance data to avoid duplication.

Self Check

Where in this folder structure would you add a new utility function to parse custom performance metrics from browser logs?

Key Result

Organize Selenium Python tests with clear layers: page objects, tests, utilities, config, and reporting for effective performance metrics collection.

Practice

1. What is the main purpose of collecting performance metrics in Selenium tests?

easy

A. To check if the page layout matches the design

B. To verify the correctness of form inputs

C. To measure how fast a web page and its resources load

D. To test user login functionality

Solution

1. Step 1: Understand performance metrics

   Performance metrics focus on timing data related to page loading and resource fetching.

2. Step 2: Identify the purpose in Selenium context

   In Selenium, performance metrics help measure loading speed, not layout or input correctness.

3. Final Answer:

   To measure how fast a web page and its resources load -> Option C

4. Quick Check:

   Performance metrics = loading speed [OK]

Hint: Performance metrics = page and resource load times [OK]

Common Mistakes:

• Confusing performance metrics with UI layout checks
• Thinking performance metrics test form validation
• Assuming performance metrics check user actions

2. Which Selenium Python code snippet correctly executes JavaScript to get the navigation start time from the performance object?

easy

A. driver.execute_script('return window.performance.timing.navigationStart;')

B. driver.execute_script('window.performance.navigationStart')

C. driver.execute_script('return window.performance.navigationStart')

D. driver.execute_script('window.performance.timing.navigationStart')

Solution

1. Step 1: Identify correct JavaScript property

   The navigation start time is under window.performance.timing.navigationStart.

2. Step 2: Check correct Selenium syntax

   Use return keyword and full property path with semicolon for valid JS execution.

3. Final Answer:

   driver.execute_script('return window.performance.timing.navigationStart;') -> Option A

4. Quick Check:

   Correct JS property and return statement = driver.execute_script('return window.performance.timing.navigationStart;') [OK]

Hint: Use full path with return and semicolon in execute_script [OK]

Common Mistakes:

• Omitting 'return' keyword in execute_script
• Using wrong property path without 'timing'
• Missing semicolon causing JS errors

3. Given the following Selenium Python code snippet, what will be the output if navigationStart is 1000 and loadEventEnd is 3500?

nav_start = driver.execute_script('return window.performance.timing.navigationStart;')
load_end = driver.execute_script('return window.performance.timing.loadEventEnd;')
load_time = load_end - nav_start
print(load_time)

medium

A. 2500

B. 4500

C. 3500

D. 1000

Solution

1. Step 1: Understand the variables

   nav_start = 1000, load_end = 3500 as given.

2. Step 2: Calculate load_time

   load_time = load_end - nav_start = 3500 - 1000 = 2500.

3. Final Answer:

   2500 -> Option A

4. Quick Check:

   3500 - 1000 = 2500 [OK]

Hint: Subtract navigationStart from loadEventEnd for load time [OK]

Common Mistakes:

• Adding instead of subtracting timestamps
• Using wrong variable names
• Confusing loadEventEnd with other timing values

4. Identify the error in this Selenium Python code snippet for collecting page load time:

nav_start = driver.execute_script('return window.performance.timing.navigationStart')
load_end = driver.execute_script('return window.performance.timing.loadEventEnd')
load_time = nav_start - load_end
print(load_time)

medium

A. Variables nav_start and load_end are not defined

B. Missing semicolons in JavaScript cause syntax error

C. Using 'driver.execute_script' is incorrect for performance data

D. The subtraction order is reversed, causing negative load time

Solution

1. Step 1: Check subtraction logic

   Load time should be load_end - nav_start, not nav_start - load_end.

2. Step 2: Understand impact of reversed subtraction

   Reversed order results in negative load time, which is incorrect.

3. Final Answer:

   The subtraction order is reversed, causing negative load time -> Option D

4. Quick Check:

   Correct subtraction order = load_end - nav_start [OK]

Hint: Subtract navigationStart from loadEventEnd, not vice versa [OK]

Common Mistakes:

• Reversing subtraction causing negative results
• Assuming missing semicolons cause errors in this context
• Thinking execute_script can't get performance data

5. You want to collect multiple performance metrics in Selenium Python: navigationStart, responseStart, and loadEventEnd. Which code snippet correctly collects these and calculates the total load time from navigationStart to loadEventEnd?

hard

A. nav = driver.execute_script('window.performance.timing.navigationStart')
resp = driver.execute_script('window.performance.timing.responseStart')
load = driver.execute_script('window.performance.timing.loadEventEnd')
total_load = resp - nav
print(total_load)

B. nav = driver.execute_script('return window.performance.timing.navigationStart;')
resp = driver.execute_script('return window.performance.timing.responseStart;')
load = driver.execute_script('return window.performance.timing.loadEventEnd;')
total_load = load - nav
print(total_load)

C. nav = driver.execute_script('return window.performance.navigationStart;')
resp = driver.execute_script('return window.performance.responseStart;')
load = driver.execute_script('return window.performance.loadEventEnd;')
total_load = load - nav
print(total_load)

D. nav = driver.execute_script('return window.performance.timing.navigationStart;')
resp = driver.execute_script('return window.performance.timing.responseStart;')
load = driver.execute_script('return window.performance.timing.loadEventEnd;')
total_load = resp - load
print(total_load)

Solution

1. Step 1: Verify JavaScript property paths

   Performance timing properties are under window.performance.timing with return keyword.

2. Step 2: Check calculation logic

   Total load time is loadEventEnd minus navigationStart, so load - nav is correct.

3. Final Answer:

   nav = driver.execute_script('return window.performance.timing.navigationStart;') resp = driver.execute_script('return window.performance.timing.responseStart;') load = driver.execute_script('return window.performance.timing.loadEventEnd;') total_load = load - nav print(total_load) -> Option B

4. Quick Check:

   Correct property path and subtraction order = nav = driver.execute_script('return window.performance.timing.navigationStart;') resp = driver.execute_script('return window.performance.timing.responseStart;') load = driver.execute_script('return window.performance.timing.loadEventEnd;') total_load = load - nav print(total_load) [OK]

Hint: Use full timing path and subtract navigationStart from loadEventEnd [OK]

Common Mistakes:

• Using wrong property paths without 'timing'
• Calculating load time with wrong subtraction order
• Omitting 'return' in execute_script calls