Category "selenium"

Preparing scraped text to mysql database in python

I ran into an issue preparing data for database, since iam doing that very first time I scraped text from html dt and dd tags, so i'am getting a lot of informa

text based HTML of the element selenium python

I am actually trying to scrap a website. In fact I have a table like this below : Table I would like to navigate to the line that contains the word "Maîtr

How select options from a smart-select?

I am trying to get all the options from a dropdown with Selenium but I get the exception that the Select module does not work with smart-select. Is there any wa

Selenium code downloads in Linux but not Windows

The following code sets up selenium with chromedriver: import os import selenium print(selenium.__version__) url = 'https://home.treasury.gov/policy-issues/fina

Python & Selenium: ElementClickInterceptedException: Message: element click intercepted error

I use Selenium in Python for scraping. The following error is displayed when I try to click button tag. ElementClickInterceptedException: Message: element click

selenium.common.exceptions.SessionNotCreatedException: Message: session not created from timeout: Timed out receiving message from renderer: 600,000

I get following error: selenium.common.exceptions.SessionNotCreatedException: Message: session not created from timeout: Timed out receiving message from render

Scrape Job description Indeed Selenium [closed]

A similar subject exists but I couldn't find the exact answer, so please could you help me? I copied from the internet the following code to s

Docker access container from java application in different container

I created a Dockerfile and i want to conntect my crawler application with my selenium server. I have connected the services via link, but its not possible to re

Continuous scraping (each second) of dynamic webpage with selenium

I am scraping website with selenium successfully using following code: import os import time from selenium import webdriver from selenium.webdriver.common.by im

How to make program (c#, selenium) wait to periodically check if the 'if' condition is met

I intend to increment the counter by 1 when the url changes and return the final count, eg 10 if url changes 10 times. The program has to wait until the user cl

Selenium and LINK_TEXT

I tried to scrapp a cost with a LINK TEXT but my scrapping method can't find the TEXT: budget = budgets.append((driver.find_element(By.LINK_TEXT, 'Total budget/

Selenium getting stuck when looping through pages

I have written code that pulls text from sites and analyses them for readability. However, I sometimes get stuck on sites. Is there any way for me to have my pr

Selenium getting stuck when looping through pages

I have written code that pulls text from sites and analyses them for readability. However, I sometimes get stuck on sites. Is there any way for me to have my pr

Selenium turning off on redirection problem

I have a problem with my selenium script. Basicly what is it supposed to do is click an element that redirects you to a completly different page (url and everyt

Django user authentication fails in selenium tests

I'm writing tests for my djangocms app, first time using selenium to replicate the browser/user. I'm trying to test the login page, user simply fills out the us

The inputs gives proper result but button doesn't work in selenium simple program

This is a simple code to add two values but whenever the code came to button it gives an error 'NoneType' object has no attribute 'click'. The values are filled

Scraping in <dt> and <dd> tags with bs4 ant python

How should i extract info i only need from <dt> and <dd> tags ? P.S and there is a lot of pages like that - hundreds Here is link for main page: ht

Java Selenium - how to test browser network -> payload?

Java Selenium - how to test browser network -> payload ? I have following code that works: WebDriver driver; BrowserMobProxyServer proxy; Proxy

How to create xpath that matches both conditions present in an element?

Description: I need to create an Xpath that matches both the condition present in the div class = 'item_inside' Explanation: Imagine There are two item_inside c

Can't get all table elements using selenium webdriver

I'm trying to get all information from this website using Python/Selenium: https://bitinfocharts.com/top-100-richest-bitcoin-addresses.html I have successfully