How can I fix "selenium.common.exceptions.NoSuchElementException"

Question

I am trying to find how many reviews a seller on bol.com, but it is saying selenium.common.exceptions.NoSuchElementException. Here is my script:

from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://www.bol.com/nl/v/daanxl/1409231/#ratings")
reviews = driver.find_element_by_xpath("/html/body/div[1]/div[2]/div/div[1]/div/div[2]/ul/li[1]/a/span").text
reviews = reviews.replace("\)", "")
reviews.replace("\(", "")
print(reviews)

Answer 1

In this case the problem is that the Xpath will return an empty elemnt so I suggest you try the Xpath or cssSelector first in the chrome console, you can use:

$x('XpathHere')

for check what an Xpath will return or

$$('ccsSelectorHere')

for check what a css selector will return.

Answer 2

To get text, first you have to wait until element will be visible. For this you can use WebDriverWait and visibility_of_element_located.
Find how to use selectors, check my example below.
You can use regular expressions to search numbers in the string. In code below, if not find any number use 0.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
import re

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 20)

driver.get("https://www.bol.com/nl/v/daanxl/1409231/#ratings")
reviews = wait.until(ec.visibility_of_element_located((By.CSS_SELECTOR, ".nav-tab__item a span"))).text
reviews_count = re.search("\\d+", reviews) if re.search("\\d+", reviews).group() else 0

Answer 3

You are seeing selenium.common.exceptions.NoSuchElementException as the locator you have used:

find_element_by_xpath("/html/body/div[1]/div[2]/div/div[1]/div/div[2]/ul/li[1]/a/span")

• Doesn't identifies the element containing the number of reviews i.e. 63.
• As per the best practices while extracting any element attribute you need to wait for the visibility of the element

---

To extract the number of reviews of a seller on bol.com i.e. 63 you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR and text attribute:

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.tst_reviews_filter_all>span"))).text)
  

• Using XPATH and get_attribute():

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@class,'tst_reviews_filter_all')]/span"))).get_attribute("innerHTML"))
  

• Console Output:

  (63)
  

---

If you want to strip the ( character from the begining and ) character from the end you can chain the the replace() method as follows:

• Using CSS_SELECTOR and text attribute:

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.tst_reviews_filter_all>span"))).text.replace('(', '').replace(')', ''))
  

• Using XPATH and get_attribute():

  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(@class,'tst_reviews_filter_all')]/span"))).get_attribute("innerHTML").replace("(", "").replace(")", ""))
  

• Console Output:

  63
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

Here you can find a detailed discussion on Selenium “selenium.common.exceptions.NoSuchElementException” when using Chrome

Answer 4

If you believe that the element will definitely exist there and you think you are having this problem just because of slow internet;

from time import sleep
def safe_sleep(function):
    while True:
       sleep(0.1)
       try:
           return function()
       except NoSuchElementException:
           continue

this function will do the job easily, it works for me.

usage;

safe_sleep(lambda: browser.find_element_by_id("foo")).send_keys("bar")

Note: If the element is not really there, the program will run forever. So don't use this method unless you're sure it exists.

Answer 5

Either your xpath is wrong or some of that element cannot may not present by that xpath.

You may use a try catch block to prevent program from stopping.

from selenium.common.exceptions import NoSuchElementException
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.bol.com/nl/v/daanxl/1409231/#ratings")
try:
    reviews = driver.find_element_by_xpath("/html/body/div[1]/div[2]/div/div[1]/div/div[2]/ul/li[1]/a/span").text
catch NoSuchElementException:
    print("Element not found")
reviews = reviews.replace("\)", "")
reviews.replace("\(", "")
print(reviews)

Answer 6

To get the no of reviews Induce WebDriverWait and element_to_be_clickable() and following xpath.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.bol.com/nl/v/daanxl/1409231/#ratings")
print(WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//a[@class='nav-tab__link tst_reviews_filter_all js_all_link'][contains(.,'Alles')]/span"))).text.replace('(','').replace(')',''))

Output on console:

63