'Python web-scraping always nothing or the main page data
I've been trying to get data from website
url = 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'
after checking the parameters, i understand I need to get this specific parameter
javax.faces.ViewState
beforehand However, my code is not working, sometimes it shows <Response [200]> but no data at all, or sometimes it show only the main page data(today's data), no matter how I changed the
formdash:bitisTarihi2_input
My code is below:
url = 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36',
'Faces-Request': 'partial/ajax',
'X-Requested-With': 'XMLHttpRequest',
'Referer': 'https://ytbsbilgi.teias.gov.tr/ytbsbilgi/frm_istatistikler.jsf'
}
# obtain view_state
vs =re.compile(r'javax.faces.ViewState:1" value="(?P<name>.*?)"', re.S)
html = urlopen(url).read()
view_state = vs.findall(str(html))[0]
Data = {
"javax.faces.partial.ajax": "true",
"javax.faces.source": "formdash:btnKabul",
"javax.faces.partial.execute": "@all",
"javax.faces.partial.render": "formdash",
"formdash:btnKabul": "formdash:btnKabul",
"formdash": "formdash",
"hidden1": "13",
"formdash:bitisTarihi2_input": "2022-05-19",
"javax.faces.ViewState": view_state
}
r = requests.post(url,data = json.dumps(Data), headers = headers, verify=False)
print(r.text)
Any suggests would be aswsome:)Thanks in advance
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
