How to convert requests.RequestsCookieJar to string

Question

I have a requests.RequestCookieJar object which contains multiple cookies from different domain/path. How can I extract a cookies string for a particular domain/path following the rules mentioned in here?

For example

>>> r = requests.get("https://stackoverflow.com")
>>> print(r.cookies)
<RequestsCookieJar[<Cookie prov=4df137f9-848e-01c3-f01b-35ec61022540 for .stackoverflow.com/>]>

# the function I expect
>>> getCookies(r.cookies, "stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"

>>> getCookies(r.cookies, "meta.stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"
# meta.stackoverflow.com is also satisfied as it is subdomain of .stackoverflow.com

>>> getCookies(r.cookies, "google.com")
""
# r.cookies does not contains any cookie for google.com, so it return empty string

Answer 1

I think you need to work with a Python dictionary of the cookies. (See my comment above.)

def getCookies(cookie_jar, domain):
    cookie_dict = cookie_jar.get_dict(domain=domain)
    found = ['%s=%s' % (name, value) for (name, value) in cookie_dict.items()]
    return ';'.join(found)

Your example:

>>> r = requests.get("https://stackoverflow.com")
>>> getCookies(r.cookies, ".stackoverflow.com")
"prov=4df137f9-848e-01c3-f01b-35ec61022540"

Answer 2

NEW ANSWER

Ok, so I still don't get exactly what it is you are trying to achieve.

If you want to extract the originating url from a requests.RequestCookieJar object (so that you could then check if there is a match with a given subdomain) that is (as far as I know) impossible.

However, you could off course do something like:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import requests
import re

class getCookies():

    def __init__(self, url):

        self.cookiejar = requests.get(url).cookies
        self.url = url

    def check_domain(self, domain):

        try:

            base_domain = re.compile("(?<=\.).+\..+$").search(domain).group()

        except AttributeError:

            base_domain = domain

        if base_domain in self.url:

            print("\"prov=" + str(dict(self.cookiejar)["prov"]) + "\"")

        else:

            print("No cookies for " + domain + " in this jar!")

Then if you do:

new_instance = getCookies("https://stackoverflow.com")

You could then do:

new_instance.check_domain("meta.stackoverflow.com")

Which would give the output:

"prov=5d4fda78-d042-2ee9-9a85-f507df184094"

While:

new_instance.check_domain("google.com")

Would output:

"No cookies for google.com in this jar!"

Then, if you (if needed) fine-tune the regex & create a list of urls, you could first loop through the list to create many instances and save them in eg a list or dict. In a second loop you could check another list of urls to see if their cookies might be present in any of the instances.

---

OLD ANSWER

The docs you link to explain:

items()

Dict-like items() that returns a list of name-value tuples from the jar. Allows client-code to call dict(RequestsCookieJar) and get a vanilla python dict of key value pairs.

I think what you are looking for is:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-

import requests

def getCookies(url):

    r = requests.get(url)

    print("\"prov=" + str(dict(r.cookies)["prov"]) + "\"")

Now I can run it like this:

>>> getCookies("https://stackoverflow.com")
"prov=f7712c78-b489-ee5f-5e8f-93c85ca06475"

Answer 3

actually , when I just have the problem as you are. but when I access the Class Define

class RequestsCookieJar(cookielib.CookieJar, MutableMapping):

I found a func called def get_dict(self, domain=None, path=None): you can simply write code like this

raw = "rawCookide"
print(len(cookie))
mycookie = SimpleCookie()
mycookie.load(raw)
UCookie={}
for key, morsel in mycookie.items():
    UCookie[key] = morsel.value

Answer 4

The following code is not promised to be "forward compatible" because I am accessing attributes of classes that were intentionally hidden (kind of) by their authors; however, if you must get into the attributes of a cookie, take a look here:

import http.cookies
import requests
import json
import sys
import os

aresponse = requests.get('https://www.att.com')
requestscookiejar = aresponse.cookies
for cdomain,cooks in requestscookiejar._cookies.items():
    for cpath, cookgrp in cooks.items():
        for cname,cattribs in cookgrp.items():
            print(cattribs.version)
            print(cattribs.name)
            print(cattribs.value)
            print(cattribs.port)
            print(cattribs.port_specified)
            print(cattribs.domain)
            print(cattribs.domain_specified)
            print(cattribs.domain_initial_dot)
            print(cattribs.path)
            print(cattribs.path_specified)
            print(cattribs.secure)
            print(cattribs.expires)
            print(cattribs.discard)
            print(cattribs.comment)
            print(cattribs.comment_url)
            print(cattribs.rfc2109)
            print(cattribs._rest)

When a person needs to access the simple attributes of cookies is it likely less complicated to go after the following way. This avoids the use of RequestsCookieJar. Here we construct a single SimpleCookie instance by reading from the headers attribute of a response object instead of the cookies attribute. The name SimpleCookie would seem to imply a single cookie but that isn't what a simple cookie is. Try it out:

import http.cookies
import requests
import json
import sys
import os

def parse_cookies(http_response):
    cookie_grp = http.cookies.SimpleCookie()
    for h,v in http_response.headers.items():
        if 'set-cookie' in h.lower():
            for cook in v.split(','):
                cookie_grp.load(cook)
    return cookie_grp

aresponse = requests.get('https://www.att.com')
cookies = parse_cookies(aresponse)
print(str(cookies))

Answer 5

You can get list of domains in ResponseCookieJar and then dump the cookies for each domain with the following code:

import requests
response = requests.get("https://stackoverflow.com")
cjar = response.cookies
for domain in cjar.list_domains(): 
    print(f'Cookies for {domain}: {cjar.get_dict(domain=domain)}')

Outputs:

Cookies for domain .stackoverflow.com: {'prov': 'efe8c1b7-ddbd-4ad5-9060-89ea6c29479e'}

In this example, only one domain is listed. It would have multiple lines in output if there were cookies for multiple domains in the Jar.

For many usecases, the cookie jar can be serialized by simply ignoring domains by calling:

dCookies = cjar.get_dict()

Answer 6

We can easily extract cookies string for a particular domain/path using functions already available in requests lib.

import requests
from requests.models import Request
from requests.cookies import get_cookie_header
session = requests.session()
r1 = session.get("https://www.google.com")
r2 = session.get("https://stackoverflow.com")
cookie_header1 = get_cookie_header(session.cookies, Request(method="GET", url="https://www.google.com"))
# '1P_JAR=2022-02-19-18; NID=511=Hz9Mlgl7DtS4uhTqjGOEolNwzciYlUtspJYxQ0GWOfEm9u9x-_nJ1jpawixONmVuyua59DFBvpQZkPzNAeZdnJjwiB2ky4AEFYVV'
cookie_header2 = get_cookie_header(session.cookies, Request(method="GET", url="https://stackoverflow.com"))
# 'prov=883c41a4-603b-898c-1d14-26e30e3c8774'

Request is used to prepare a :class:PreparedRequest <PreparedRequest>, which is sent to the server.

Answer 7

What you need is get_dict() method

a_session = requests.Session()
a_session.get('https://google.com/')
session_cookies = a_session.cookies
cookies_dictionary = session_cookies.get_dict()

# Now just print it or convert to json
as_string = json.dumps(cookies_dictionary)
print(cookies_dictionary)