'Inject javascript fetch and XHR into Selenium with Python
There are several ways to scrape information from websites. Recently I've heard that you can open a selenium webdriver and inject JS on it to fetch specific url from within the website or to replicate AJAX calls.
For example
driver.execute_script("return fetch('www.yoururlhere')")
So, I have a couple of questions regarding it
1 - How do you actually fetch the answer? Inspecting the driver I can see the call but neither in the frontpage nor doing driver.page_source am I able to retrieve it.
2 - How would one replicate more complex requests calls where there is params or data?
requests.get(url=url,headers=headers,params=params)
requests.post(url=url,headers=headers, data=json.loads(data))
How does this work?
Thanks!
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
