'Issue facing while webscraping yell.com using cloudscrapper

I was trying to scrape the yell.com. My code was running smoothly till last month after they have increased the security. Now they are using some updated version of cloudflare. My previous code :

import cloudscraper

headers = {'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
#'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-IN,en;q=0.9',
'cache-control': 'no-cache',
'pragma': 'no-cache',
'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"',
'sec-ch-ua-mobile': '?0',
'sec-ch-ua-platform': '"Windows"',
'sec-fetch-dest': 'document',
'sec-fetch-mode': 'navigate',
'sec-fetch-site': 'none',
'sec-fetch-user': '?1',
'upgrade-insecure-requests': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.54 Safari/537.36',}

scraper = cloudscraper.create_scraper(allow_brotli = False , disableCloudflareV1 = True)
query ="https://www.yell.com/ucs/UcsSearchAction.do?keywords=plumber&location=Wakefield"
req = scraper.get(query,headers=headers)
print(req)
res= req.text
file=open("Checking_1.html","w",encoding="utf-8")
file.write(res)
file.close()            

But I am getting a 403 error code now. and getting a captcha.

And when I used debug mode, I was getting these in response:

enter image description here

Please let me know if i am making some mistakes.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source