Network log capture in Selenium Python

Introduction

Network log capture helps you see what data your browser sends and receives during testing. It shows requests and responses, so you can check if your app talks to servers correctly.

You want to check if a web page loads all images and scripts properly.

You need to verify that API calls return the right data during a test.

You want to find out why a page is slow by looking at network requests.

You want to confirm that sensitive data is not sent over insecure connections.

Syntax

Selenium Python

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

caps = DesiredCapabilities.CHROME.copy()
caps['goog:loggingPrefs'] = {'performance': 'ALL'}

options = Options()
service = Service('/path/to/chromedriver')
driver = webdriver.Chrome(service=service, options=options, desired_capabilities=caps)

# After loading a page
logs = driver.get_log('performance')

Use desired_capabilities to enable performance logging in Chrome.

Logs contain network events in JSON format inside the 'message' field.

Examples

This sets Chrome to capture all performance logs, including network events.

Selenium Python

caps = DesiredCapabilities.CHROME.copy()
caps['goog:loggingPrefs'] = {'performance': 'ALL'}

This retrieves and prints all performance log messages from the browser.

Selenium Python

logs = driver.get_log('performance')
for entry in logs:
    print(entry['message'])

Sample Program

This script opens a browser, navigates to example.com, captures network responses, and prints each URL with its HTTP status code.

Selenium Python

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import json

caps = DesiredCapabilities.CHROME.copy()
caps['goog:loggingPrefs'] = {'performance': 'ALL'}

options = Options()
service = Service('/path/to/chromedriver')
driver = webdriver.Chrome(service=service, options=options, desired_capabilities=caps)

try:
    driver.get('https://example.com')
    logs = driver.get_log('performance')
    network_events = []
    for entry in logs:
        message = json.loads(entry['message'])['message']
        if 'Network.responseReceived' == message['method']:
            url = message['params']['response']['url']
            status = message['params']['response']['status']
            network_events.append(f"URL: {url}, Status: {status}")
    for event in network_events:
        print(event)
finally:
    driver.quit()

Important Notes

Make sure to use the correct path to your ChromeDriver in Service.

Network logs can be large; filter them to find useful info.

Not all browsers support performance logging the same way; Chrome is recommended.

Summary

Network log capture lets you see browser-server communication during tests.

Enable performance logging in Chrome to get network events.

Parse logs to check URLs and response status codes for validation.

Practice

1. What is the main purpose of capturing network logs in Selenium tests?

easy

A. To see the communication between browser and server during tests

B. To speed up the test execution time

C. To change the browser's UI elements

D. To generate random test data automatically

Solution

1. Step 1: Understand network logs in Selenium

   Network logs capture the details of requests and responses between the browser and server during test execution.

2. Step 2: Identify the main purpose

   This helps testers verify if the correct URLs are called and if responses are successful.

3. Final Answer:

   To see the communication between browser and server during tests -> Option A

4. Quick Check:

   Network logs show browser-server communication [OK]

Hint: Network logs = browser-server communication details [OK]

Common Mistakes:

• Thinking network logs speed up tests
• Confusing UI changes with network logs
• Assuming network logs generate test data

2. Which Chrome option enables performance logging to capture network events in Selenium?

easy

A. "args": ["--disable-extensions"]

B. "goog:loggingPrefs": {"performance": "ALL"}

C. "prefs": {"download.default_directory": "/tmp"}

D. "chromeOptions": {"headless": true}

Solution

1. Step 1: Identify the correct Chrome option for logging

   Performance logging is enabled by setting "goog:loggingPrefs" with "performance" set to "ALL".

2. Step 2: Understand other options

   Other options like headless mode or disabling extensions do not enable network log capture.

3. Final Answer:

   "goog:loggingPrefs": {"performance": "ALL"} -> Option B

4. Quick Check:

   Enable performance logs with goog:loggingPrefs [OK]

Hint: Use goog:loggingPrefs with performance ALL to capture network logs [OK]

Common Mistakes:

• Confusing headless mode with logging
• Using download preferences for network logs
• Disabling extensions instead of enabling logs

3. Given this code snippet, what will be printed?

logs = driver.get_log('performance')
for entry in logs:
    message = json.loads(entry['message'])['message']
    if message['method'] == 'Network.responseReceived':
        print(message['params']['response']['url'])

medium

A. All browser console error messages

B. Only URLs of failed network requests

C. All URLs of network responses received during the test

D. List of all clickable elements on the page

Solution

1. Step 1: Understand the log parsing code

   The code reads performance logs, filters for 'Network.responseReceived' events, and prints the response URLs.

2. Step 2: Identify what is printed

   It prints all URLs for network responses received, regardless of success or failure.

3. Final Answer:

   All URLs of network responses received during the test -> Option C

4. Quick Check:

   Code prints URLs from Network.responseReceived events [OK]

Hint: Network.responseReceived logs contain all response URLs [OK]

Common Mistakes:

• Assuming only failed URLs are printed
• Confusing network logs with console errors
• Thinking it lists page elements

4. What is wrong with this code snippet for capturing network logs?

options = webdriver.ChromeOptions()
options.set_capability('loggingPrefs', {'performance': 'ALL'})
driver = webdriver.Chrome(options=options)
logs = driver.get_log('performance')

medium

A. The capability key should be 'goog:loggingPrefs', not 'loggingPrefs'

B. The ChromeOptions object cannot be used with Chrome driver

C. The get_log method does not exist in Selenium

D. Performance logs cannot be captured in Chrome

Solution

1. Step 1: Check capability key correctness

   The correct key to enable performance logging is 'goog:loggingPrefs', not 'loggingPrefs'.

2. Step 2: Verify other statements

   ChromeOptions can be used with Chrome driver, get_log exists, and performance logs can be captured.

3. Final Answer:

   The capability key should be 'goog:loggingPrefs', not 'loggingPrefs' -> Option A

4. Quick Check:

   Use 'goog:loggingPrefs' to enable performance logs [OK]

Hint: Use 'goog:loggingPrefs' key for performance logging [OK]

Common Mistakes:

• Using wrong capability key
• Thinking ChromeOptions is incompatible
• Believing get_log method is missing

5. How can you verify in Selenium that a specific API call to 'https://api.example.com/data' returned status code 200 using network logs?

hard

A. Use driver.execute_script to intercept network calls directly

B. Use driver.get_log('browser') and search for the URL string

C. Check page source for the URL and assume status 200 if found

D. Parse performance logs, find 'Network.responseReceived' events, check if response URL matches and status is 200

Solution

1. Step 1: Capture and parse performance logs

   Enable performance logging and get logs with driver.get_log('performance').

2. Step 2: Filter 'Network.responseReceived' events and check URL and status

   Look for events where response URL matches 'https://api.example.com/data' and verify status code is 200.

3. Final Answer:

   Parse performance logs, find 'Network.responseReceived' events, check if response URL matches and status is 200 -> Option D

4. Quick Check:

   Check responseReceived events for URL and status 200 [OK]

Hint: Filter Network.responseReceived logs by URL and status code [OK]

Common Mistakes:

• Using browser logs instead of performance logs
• Assuming URL presence means status 200
• Trying to intercept network calls with execute_script