'Python web-scraping always nothing or the main page data

I've been trying to get data from website

url = 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'

after checking the parameters, i understand I need to get this specific parameter

javax.faces.ViewState

beforehand However, my code is not working, sometimes it shows <Response [200]> but no data at all, or sometimes it show only the main page data(today's data), no matter how I changed the

formdash:bitisTarihi2_input

My code is below:

url = 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36',
    'Faces-Request': 'partial/ajax',
    'X-Requested-With': 'XMLHttpRequest',
    'Referer': 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'
}

# obtain view_state
vs =re.compile(r'javax.faces.ViewState:1" value="(?P<name>.*?)"', re.S)
html = urlopen(url).read()
view_state = vs.findall(str(html))[0]

Data = {
    "javax.faces.partial.ajax": "true",
    "javax.faces.source": "formdash:btnKabul",
    "javax.faces.partial.execute": "@all",
    "javax.faces.partial.render": "formdash",
    "formdash:btnKabul": "formdash:btnKabul",
    
    "formdash": "formdash",
    "hidden1": "13",
    "formdash:bitisTarihi2_input": "2022-05-19",
    "javax.faces.ViewState": view_state
}

r = requests.post(url,data = json.dumps(Data), headers = headers, verify=False)

print(r.text)

Any suggests would be aswsome:)Thanks in advance



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source