'Clicking Next on webpage with Selenium/Python

I'm trying to use Selenium to click Next on https://www.theitdepot.com/products-Motherboards_C13.html

The relevant HTML code is:

<li class="page-item mx-1">
<a class="page-link rounded-pill border-0 bg-transparent text-yellow category_page_class" href="#" id="2">Next →</a>
</li>

So far, I have tried two ways of clicking Next unsuccessfully:

  1. Using .click()

    from selenium import webdriver
    from webdriver_manager.chrome import ChromeDriverManager
    from selenium.webdriver.common.by import By
    
    URL = "https://www.theitdepot.com/products-Motherboards_C13.html"
    
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-dev-shm-usage')
    wd = webdriver.Chrome(ChromeDriverManager().install(), options=options)
    wd.get(URL)
    
    next = wd.find_element(by=By.XPATH, value="//*[contains(text(), 'Next →')]") 
    next.click() 
    time.sleep(10)
    
  2. Using execute_script

    next = wd.find_element(by=By.XPATH, value="//*[contains(text(), 'Next →')]") 
    wd.execute_script("arguments[0].click();", next) 
    

In both cases, while I do not get any error, the Next button does not actually get clicked, and the rest of my code loops just execute over the same information as on Page 1.

Is there anything obvious I'm missing, or any other ways to click the Next button that would work on this webpage?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source