'Guzzle request agaisn't cloudflare protected server - works from development env but not production
I've got a bit of a strange one. I've been scraping a website for a while then I think they protected it with cloudflare which resulted in me getting a '...resulted in a 403 Forbidden response'.
So I added a user agent in and rotated my proxies and it works from my development machine however when deploying to my live server I am still getting a 403 error. Same code, same user agent, same proxy IP.
My code is below:
$client = new Client();
$response = $client->request('GET', 'https://www.targeturl.com', ['proxy' => '1.2.3.4:5432', 'headers' => [
'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.37',
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'Accept-Encoding' => 'gzip, deflate, br',
]]);
dd($response->getBody()->getContents());
Any ideas why?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
