'"HTTPConnectionPool(host='127.0.0.1', port=64803): Max retries exceeded with url" error in selenium Python
I'm in the middle of programming a selenium bot that will scrape.
A browser is created in each for loop. Then at the end of the loop, a new Thread starts by calling a function.
for i in range(10):
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument('--ignore-certificate-errors')
options.add_argument("--disable-webgl")
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option("detach", True)
prefs = {"credentials_enable_service": False,
"profile.password_manager_enabled": False}
options.add_experimental_option("prefs", prefs)
options.add_argument('disable-notifications')
options.add_argument('disable-infobars')
driver = webdriver.Chrome(options=options)
threading.Thread(target=NextStep).start()
And in this function:
def NextStep():
driver2 = create_driver_session(session_id=session_id, executor_url=executor_url)
WebDriverWait(driver2, 300).until(EC.text_to_be_present_in_element((By.XPATH, '//*[@id="main"]/div/div[2]/div[2]/footer/div/div[2]/div/div[2]/div[1]'), chrono))
create_driver_session:
def create_driver_session(session_id, executor_url):
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
# Save the original function, so we can revert our patch
org_command_execute = RemoteWebDriver.execute
def new_command_execute(self, command, params=None):
if command == "newSession":
# Mock the response
return {'success': 0, 'value': None, 'sessionId': session_id}
else:
return org_command_execute(self, command, params)
# Patch the function before creating the driver object
RemoteWebDriver.execute = new_command_execute
new_driver = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
new_driver.session_id = session_id
# Replace the patched function with original function
RemoteWebDriver.execute = org_command_execute
return new_driver
The function attaches a chrome session for each browser created.
After the 3rd browser an error appears:
File "C:\Users\33782\anaconda3\lib\site-packages\urllib3\util\retry.py", line 574, in
increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=64803): Max
retries exceeded with url: /session/61a95bc6911f9d4fcd0549782de3cf60/element (Caused by
NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000002515ADD8700>:
Failed to establish a new connection: [WinError 10061]
And webdriver.remote disconnects from the browser.
Any help would be appreciated, I can't sleep anymore :)
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
