'Selenium popup data extraction using Python
I am a Selenium beginner looking to extract data from popup windows (HTML) via Selenium and Python.
This is the website: https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y
I am trying to click on the "detail" popup for every single Company and extract all of the data. This is what I have so far (sorry, not much!) which simply clicks the first "detail" popup.
#import packages
from selenium import webdriver
#create filepath for correct chrome webdriver
path = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(path)
driver.get("https://www.drayage.com/directory/results.cfm?city=SAV&port=y&OceanCntrs=y&drvrs=y&showClicks=y")
driver.find_element_by_link_text("detail").click()
If I can get a text block for each of the popups that would even be enough for me to then separate via delimiters and clean in SQL.
My trouble is how to extract all of this as well as how to continually click each "detail" popup.
Thank you so much in advance for the help!
Update 1
Thanks for the comments so far!
Once I get into the popup window by clicking "detail" I am trying to extract everything from the first table which has the XPATH "/html/body/table[1]".
Unfortunately the below code isn't working. Let me know if you are able to help and thank you again!
table = driver.find_element_by_xpath("/html/body/table[1]")
for i in driver.find_elements_by_xpath("/html/body/table[1]/tbody"):
data = [table.text for table in i.find_elements_by_xpath(".//*[self::td]")]
print(data)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
