Puppeteer in NodeJS reports 'Error: Node is either not visible or not an HTMLElement'

Question

I'm using 'puppeteer' for NodeJS to test a specific website. It seems to work fine in most case, but some places it reports:

Error: Node is either not visible or not an HTMLElement

The following code picks a link that in both cases is off the screen.

The first link works fine, while the second link fails.

What is the difference? Both links are off the screen.

Any help appreciated, Cheers, :)

Example code

const puppeteer = require('puppeteer');

const initialPage = 'https://statsregnskapet.dfo.no/departementer';
const selectors = [
    'div[id$="-bVMpYP"] article a',
    'div[id$="-KcazEUq"] article a'
];

(async () => {
    let selector, handles, handle;
    const width=1024, height=1600;
    const browser = await puppeteer.launch({ 
        headless: false, 
        defaultViewport: { width, height } 
    });
    const page = await browser.newPage();
    await page.setViewport({ width, height});
    page.setUserAgent('UA-TEST');

    // Load first page
    let stat = await page.goto(initialPage, { waitUntil: 'domcontentloaded'});

    // Click on selector 1 - works ok
    selector = selectors[0];
    await page.waitForSelector(selector);
    handles = await page.$$(selector);
    handle = handles[12]
    console.log('Clicking on: ', await page.evaluate(el => el.href, handle));
    await handle.click();  // OK

    // Click that selector 2 - fails
    selector = selectors[1];
    await page.waitForSelector(selector);
    handles = await page.$$(selector);
    handle = handles[12]
    console.log('Clicking on: ', await page.evaluate(el => el.href, handle));
    await handle.click();  // Error: Node is either not visible or not an HTMLElement

})();

I'm trying to emulate the behaviour of a real user clicking around the site, which is why I use .click(), and not .goto(), since the a tags have onclick events.

Answer 1

If your code looks like this

const button = await page.$(selector);
await button.click();

Try changing

await button.click();

to

await button.evaluate(b => b.click());

The difference is that button.click() clicks using Puppeteer's ElementHandle.click() which

1. scrolls the page until the element is in view
2. gets the bounding box of the element (this step is where the error happens) and finds the screen x and y pixel coordinates of the middle of that box
3. moves the virtual mouse to those coordinates and sets the mouse to "down" then back to "up", which triggers a click event on the element under the mouse

whereas button.evaluate(b => b.click()) "clicks" the element by running the JavaScript HTMLElement.click() method on the given element in the browser context, which fires a click event. It doesn't scroll the page or move the mouse and works even if the element is off-screen.

Answer 2

I know I’m late to the party but I discovered an edge case that gave me a lot of grief, and this thread, so figured I’d post my findings.

The culprit: CSS

scroll-behavior: smooth

If you have this you will have a bad time.

The solution:

await page.addStyleTag({ content: "{scroll-behavior: auto !important;}" });

Hope this helps some of you.

Answer 3

For anyone still having trouble this worked for me:

await page.evaluate(()=>document.querySelector('#sign-in-btn').click())

Basically just get the element in a different way, then click it.

The reason I had to do this was because I was trying to click a button in a notification window which sits outside the rest of the app (and Chrome seemed to think it was invisible even if it was not).

Answer 4

My way

async function getVisibleHandle(selector, page) {

    const elements = await page.$$(selector);

    let hasVisibleElement = false,
        visibleElement = '';

    if (!elements.length) {
        return [hasVisibleElement, visibleElement];
    }

    let i = 0;
    for (let element of elements) {
        const isVisibleHandle = await page.evaluateHandle((e) => {
            const style = window.getComputedStyle(e);
            return (style && style.display !== 'none' &&
                style.visibility !== 'hidden' && style.opacity !== '0');
        }, element);
        var visible = await isVisibleHandle.jsonValue();
        const box = await element.boxModel();
        if (visible && box) {
            hasVisibleElement = true;
            visibleElement = elements[i];
            break;
        }
        i++;
    }

    return [hasVisibleElement, visibleElement];
}

Usage

let selector = "a[href='https://example.com/']";

let visibleHandle = await getVisibleHandle(selector, page);

if (visibleHandle[1]) {

   await Promise.all([
     visibleHandle[1].click(),
     page.waitForNavigation()
   ]);
}