'Problem with parsing on selenium, scrolling

there is a problem, I will explain the script of the code.

First, we go to the main page, then the for loop that goes to page 1, scrolls and parses the links to the photo (it doesn’t parse without scrolling), then goes to page 2, and there is already a problem. It scrolls to the bottom in a second. And I need it to scroll gradually, like on the first page. I throw the code:

driver.get(url=url)

n = 0

urla = driver.find_element(By.CLASS_NAME, "ipsPagination_pageJump").text


for page_number in range(int(urla.split()[3])):
    page_number = page_number + 1
    driver.get(url=url + f"page/{page_number}")
    time.sleep(2)
    imgs = driver.find_elements(By.CLASS_NAME, "cGalleryPatchwork_image")
    for i in imgs:
        n = n + 500
        driver.execute_script(f"window.scrollTo(0, {n})")
        time.sleep(0.2)
        print(i.get_attribute("src"))

driver.quit()

I know the code is very bad and not optimized



Solution 1:[1]

to scroll gradually one element by one element you should use the following execute_script command driver.execute_script("arguments[0].scrollIntoView(true);", i)

Code:

for page_number in range(int(urla.split()[3])):
    page_number = page_number + 1
    driver.get(url=url + f"page/{page_number}")
    time.sleep(2)
    imgs = driver.find_elements(By.CLASS_NAME, "cGalleryPatchwork_image")
    for i in imgs:
        #n = n + 500
        #driver.execute_script(f"window.scrollTo(0, {n})")
        driver.execute_script("arguments[0].scrollIntoView(true);", i)
        time.sleep(0.2)
        print(i.get_attribute("src"))

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 cruisepandey