'cURL request with 'User-Agent' header works fine but doesnt work in node with axios/fetch/https
When i make a curl request to a website for webscraping with the header "User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:93.0) Gecko/20100101 Firefox/93.0" it works with status code 200. But when i use it in node js with axios or https or fetch, i get a 403 error. What's going wrong? i cant figure it out?
const response = await fetch(url, {
headers : {
"User-Agent" : "Mozilla/5.0 (X11; Linux x86_64; rv:93.0) Gecko/20100101 Firefox/93.0"
}
});
return response;
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
