'In webscraping and saving request header information should I be concerned about them changing/breaking the script?
I am creating a script to scrape data off of a website (POST request) and these are the header information I'm saving to make the script work.
headers = {
'User-Agent': "",
'Accept': "",
'Accept-Language': "",
'Accept-Encoding': "",
'Referer': "",
'Content-Type': "",
'Access-Control-Allow-Methods': "",
'Access-Control-Allow-Origin': "",
'Access-Control-Allow-Headers': "",
'Cache-Control': "",
'Pragma': "",
'Expires': "",
'X-Frame-Options': "",
'X-Content-Type-Options': "",
'env': "",
'type': "",
'token': "",
'clientId': "",
'Origin': "",
'DNT': "",
'Connection': "",
'Sec-Fetch-Dest': "",
'Sec-Fetch-Mode': "",
'Sec-Fetch-Site': "",
'TE': ""
}
Is [token] based on session, hardware, or what?
Should I be concerned about the information in headers to change and breaking the script or can I safely set and forget?
Thank you.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
