'How to receive a json response using proxycrawl
from proxycrawl import CrawlingAPI
from bs4 import BeautifulSoup
api = CrawlingAPI({'token': 'wMtzOpKeL9CCASWQBGiRLA'})
# a = """GET 'https://api.proxycrawl.com/?token=wMtzOpKeL9CCASWQBGiRLA&url=https%3A%2F%2Fwww.amazon.com&format=json'
# Response:
# {
# "original_status": "200",
# "pc_status": 200,
# "url": "https://www.jackson.org/provider-directory/?specialty=Allergy",
# "body": "\u003C!doctype html\u003E\u003Chtml class=\"a-no-js\" data-19ax5a9jf\n... (all the html of the page)"
# }"""
targetURL = 'https://www.jackson.org/provider-directory/?specialty=Allergy'
response = api.get(targetURL)
if response['status_code'] == 200:
b_soup = BeautifulSoup(response['body'], 'lxml')
print(b_soup)
Is there anyway to receive the json response using proxycrawling and get the details from the webpage. Link: https://proxycrawl.com/docs/crawling-api/response/#html-response
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
