How to Handle Cookies in Python Requests: Simple Guide

To handle cookies in python requests, use the cookies parameter to send cookies and the response.cookies attribute to read cookies from a response. For persistent cookie handling across requests, use a requests.Session() object which automatically manages cookies.

Why This Happens

When you try to send or receive cookies using the requests library without managing them properly, cookies may not be sent or saved as expected. This happens because each requests.get() or requests.post() call is independent and does not remember cookies from previous responses.

python

import requests

# Trying to send cookies without session
cookies = {'session_id': '12345'}
response = requests.get('https://httpbin.org/cookies', cookies=cookies)
print(response.text)

# Trying to get cookies from response but not saving them
response2 = requests.get('https://httpbin.org/cookies/set?mycookie=value')
print(response2.text)

response3 = requests.get('https://httpbin.org/cookies')
print(response3.text)

Output

{
  "cookies": {
    "session_id": "12345"
  }
}

{
  "cookies": {
    "mycookie": "value"
  }
}

{
  "cookies": {}
}

The Fix

Use a requests.Session() object to persist cookies across multiple requests. This session object automatically saves cookies sent by the server and sends them back in subsequent requests.

python

import requests

session = requests.Session()

# Set a cookie on the server
response1 = session.get('https://httpbin.org/cookies/set?mycookie=value')
print('After setting cookie:', response1.text)

# Now get cookies stored in the session
response2 = session.get('https://httpbin.org/cookies')
print('Cookies sent back:', response2.text)

Output

After setting cookie: {
  "cookies": {
    "mycookie": "value"
  }
}
Cookies sent back: {
  "cookies": {
    "mycookie": "value"
  }
}

Prevention

Always use a requests.Session() when you need to maintain cookies across multiple requests, such as logging into a website or tracking sessions. Avoid manually managing cookies unless you have a specific reason. This approach prevents losing cookies and makes your code cleaner and more reliable.

Also, inspect response.cookies to debug what cookies the server sends back.

Related Errors

Cookies not sent: Happens if you forget to use a session or pass cookies explicitly.

Cookies not saved: Happens if you do not use a session and expect cookies to persist between requests.

Incorrect cookie format: Cookies must be a dictionary when passed to cookies= parameter.

Key Takeaways

Use requests.Session() to automatically handle cookies across requests.

Pass cookies as a dictionary with the cookies= parameter for single requests.

Check response.cookies to see cookies sent by the server.

Avoid making separate requests without a session if cookies need to persist.

Sessions simplify cookie management and reduce bugs in HTTP requests.