'Python requests behind proxy blocked in AWS EC2 but not in local

I am trying to scrap a page on tickpick.com with AWS EC2. For this I use the python requests module. I was suspecting tickpick.com to block AWS IP addresses, therefore I set up a free proxy in order to mask the AWS IP address. I also tried setting up an user agent.

I'm having a hard time because the python script request is blocked by Cloudfare when run on AWS EC2 but not when it's run on my local machine (Windows 10).

import requests

URL_MAIN = 'https://www.tickpick.com/mlb/los-angeles-dodgers-tickets/?page=1'
proxies = {
    'http': 'http://*.*.*.*:80' # free proxy
}

r = requests.get(URL_MAIN, proxies = proxies, headers={'User-Agent': 'Chrome'})

print(r.content)

I have no clue why it would run in local and not in EC2 as I use the exact same python script and proxy to connect. I don't want to use selenium because it is too slow for my application. Is there anything else I can do ?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source