'AWS API Gateway 301 when call Instagram
I'm trying to fetch some data from Instagram and to prevent IP limitation I want to use AWS API Gateway proxies. I'm using a python lib requests_ip_rotator to manage my gateway and it works well, my requests on few website get a 200. But when I make a request on Instagram, all my requests are redirected with a HTTP 301 Response.
Here is my code, pretty simple, you can remove the mount to check that the request works well without the gateway setup.
import requests
from requests_ip_rotator import ApiGateway
gateway = ApiGateway("https://www.instagram.com/", regions=['eu-west-3'], access_key_id=AWS_ID,
access_key_secret=AWS_KEY)
gateway.start()
session = requests.Session()
session.mount("https://www.instagram.com/", gateway)
response = session.get("https://www.instagram.com/neymarjr/feed")
print(response)
gateway.shutdown()
Hope someone can help me ! If you need more information don't hesitate !
And feel free to give me solution for mass "scrapping" on Instagram =)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
