'Can't find button by CSS Selenium
Goal
I'm trying to create a list of source image URLs for the first n images in google search results, in this case for the search term "cat"
Approach
I'm doing this with Selenium and Google Chrome. My method is
- Search the term
- Click on the first image. This pops up a bigger preview
- Get the source URL of this image and write it to a file
- Click the "next" button in the popup image
- Repeat steps 3 & 4 for n-1 more times
Problem
Everything works fine until the second time I try to click the "next" button, and I'm not exactly sure what the problem is. I tried debugging, finding the element by XPATH multiple ways, finding it by CSS selector multiple ways, but I can't figure it out for the life of me.
Has anybody dealt with any issues like this? Or perhaps have a smarter way of doing what I'm trying to accomplish? Thanks in advance!
Code
# Function parameters
timeout_delay = 20
full_size_delay = 0.1
n=10
# Search "cat" in google image search
url = f"https://www.google.com/search?q=cat&tbm=isch"
self.wd.get(url)
# Find the first image and click on it
# self.wd.find_element
first_image = WebDriverWait(self.wd,
timeout_delay).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".rg_i.Q4LuWd")))
first_image.click()
# Open up a file so we can write to it
with open(f'cat.txt', 'a') as f:
# For as many images as we elect to save
for i in range(n):
time.sleep(0.5)
# Find the full-size popup
full_size = self.wd.find_elements(By.CLASS_NAME, "n3VNCb")
# Find the associated URL of the image and write it to the file
for j in full_size:
time.sleep(full_size_delay)
pic_url = j.get_attribute("src")
if "http" in pic_url:
#print(pic_url)
f.write(pic_url + "\n")
break
# Find the "next image" right arrow button in the popup to go to the next image
#next_button = self.wd.find_element(By.CSS_SELECTOR, ".knIqbf")
next_button = WebDriverWait(self.wd, 5).until(
EC.element_to_be_clickable((By.CSS_SELECTOR, "a.knIqbf.SIwKhe")))
#next_button = self.wd.find_element(By.XPATH, "a[jsname='gxjVle']")
next_button.click()
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
