Intermittent Stale Element Reference Exception

Question

I am writing a python selenium script to scrape data from a website. The script reads the employee identification number, name and date of birth from a CSV file and copies it to a Dictionary & then enters data from each line into a form and then clicks a submit button. The next page that appears asks for the years of employment history which defaults to one year which is fine for my use case so the only thing the script does on this page is click the search button. The script works consistently for approximately 20 to 35 employees. But at some point the script will fail with the following stale element reference exception:

Message: The element reference of is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

The relevant HTML for the Search button is:

<div data-container="" class="btn-label OSInline">Search</div>
</button>
</div>
</div>
</div><div data-container="" style="text-align: center;" id="b8-ResetFilters2"><a data-link="" href="#" style="width: auto;"><span data-expression="" style="width: auto;">Reset Search</span></a>
</div>
</div>
</div>
<div

The relevant Python Selenium code is (GO DOWN TO "*****" TO SEE WHERE I THE SEARCH BUTTON IS):

# Open MOCemployees.csv as readable file & then convert to a Python Dictionary
with open('MOCemployees.csv', 'r') as MOCemployees.:
    MOCemployees.Dict = csv.DictReader(MOCemployees.)

    for line in MCpatientsDict:
        MOCEmployeeNumber = (line['MCID'])
        LastName = (line['LastName'])
        FirstName = (line['FirstName'])
        DOB = (line['DOB'])

# Explicit Wait for MOCEmployeeNumberInput then click on it and enter MOCEmployeeNumber
        MOCEmployeeNumberInput = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.ID, 'b7-MOCEmployeeNumberInput')))
# Locate LastNameInput and click on it and enter LastName
        LastNameInput = driver.find_element(By.ID, 'b7-LastNameInput')
        LastNameInput.click()
        LastNameInput.send_keys(LastName)
# Locate FirstNameInput and click on it and enter FirstNameI
        FirstNameInput = driver.find_element(By.ID, 'b7-FirstNameInput')
        FirstNameInput.click()
        FirstNameInput.send_keys(FirstName)
# Locate DOBInput and click on it and enter DOB
        DOBInput = driver.find_element(By.ID, 'b7-DOBInput')
        DOBInput.click()
#DOBInput.send_keys('YYYY-MM-DD')
        DOBInput.send_keys(DOB)

# Explicit Wait for Submit Button & click on it
        SubmitButton = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, 'button.margin-top-m')))
        SubmitButton.click()

# *****Explicit Wait for Search Button & click on it to accept default 1 year of data *****THIS BUTTON IS NOT CLICKED INTERMITTENTLY AND THE STALE ELEMENT REFERENCE EXCEPTION OCCURS AT THIS POINT*****
        SearchButton = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.btn-label')))
        SearchButton.click()
#Wait
        time.sleep(3)

# print MOCID, NAME & then the value of the value attribute 
        RemainDeductible = (WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "b8-b36-Input_RemainAmtYr1"))).get_attribute("value"))

#Save MOCID, Name & Remaining Due to a CSV file
        POutput = ','.join((MOCEmployeeNumber, LastName, FirstName, DOB, RemainDeductible)) + '\n'

#Locate Change Employee Link & click on it
        SelectButton = driver.find_element(By.CSS_SELECTOR, '#b7-b3-Column4 > div:nth-child(1) > a:nth-child(1) > span:nth-child(1)')
        SelectButton.click()

How can I prevent this stale element reference exception?

Answer 1

Maybe the element is not accessible at the very first attempt in the HTML DOM.

You can try to perform click again in a loop like this:

Code:

# *****Explicit Wait for Search Button & click on it to accept default 1 year of data *****THIS BUTTON IS NOT CLICKED INTERMITTENTLY AND THE STALE ELEMENT REFERENCE EXCEPTION OCCURS AT THIS POINT*****
attempts = 0
while attempts < 2:
    try:
        SearchButton = WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.btn-label')))
        SearchButton.click()
        break
    except StaleElementReferenceException as e:
        print('')
       # print('Could not click', e.msg)
    attempts = attempts  + 1

You can probably try with while attempts < 5: