Network log capture in Selenium Python - Framework Patterns

Framework Mode - Network log capture

Folder Structure

network_log_capture_project/
├── tests/
│   ├── test_login.py
│   └── test_api_calls.py
├── pages/
│   ├── login_page.py
│   └── dashboard_page.py
├── utils/
│   ├── network_logger.py
│   └── helpers.py
├── config/
│   ├── config.yaml
│   └── credentials.yaml
├── conftest.py
└── requirements.txt

Test Framework Layers

• Driver Layer: Manages Selenium WebDriver setup and teardown, including enabling network logging capabilities.
• Page Objects: Encapsulate UI elements and actions for each page (e.g., LoginPage, DashboardPage).
• Tests: Test cases that use page objects and utilities to perform actions and assertions.
• Utilities: Helper modules like network_logger.py to capture and process network logs.
• Configuration: YAML files for environment settings, browser options, and credentials.
• Fixtures (conftest.py): Pytest fixtures to initialize WebDriver with network logging enabled and provide reusable setup.

Configuration Patterns

• Environment Settings: Use config/config.yaml to define URLs, timeouts, and environment-specific variables.
• Browser Options: Configure Chrome or Firefox to enable performance logging for network capture via Selenium capabilities.
• Credentials: Store sensitive data like usernames and passwords securely in config/credentials.yaml, avoid hardcoding.
• Dynamic Loading: Use conftest.py to load config files and pass settings to tests and utilities.

Test Reporting and CI/CD Integration

• Test Reports: Use Pytest with plugins like pytest-html to generate readable reports including test results and captured network logs summaries.
• Network Logs Storage: Save network logs as JSON files per test run for debugging and audit purposes.
• CI/CD Integration: Integrate tests in pipelines (e.g., GitHub Actions, Jenkins) to run on pull requests and merges, collecting logs and reports as artifacts.
• Alerts: Configure notifications on test failures with links to logs and reports for quick triage.

Best Practices

1. Use Browser Performance Logging: Enable browser performance logs via Selenium capabilities to capture network traffic reliably.
2. Separate Concerns: Keep network log capture logic in utility modules, not mixed with test or page object code.
3. Manage Sensitive Data Securely: Never hardcode credentials; use config files with proper access control.
4. Explicit Waits: Use explicit waits to ensure network requests complete before capturing logs.
5. Clean Up Resources: Properly close WebDriver sessions to avoid resource leaks and stale logs.

Self Check

Where in this folder structure would you add a new utility to parse and filter network logs for specific API endpoints?

Key Result

Enable browser performance logging via Selenium capabilities and organize network log capture in utility modules for clean, maintainable test automation.