Category "puppeteer"

How will I get the content ie the title of a tag while web scrapping with puppeteer?

I am following this tutorial. https://www.digitalocean.com/community/tutorials/how-to-scrape-a-website-using-node-js-and-puppeteer I am trying to learn how to u

Scroll inside div doesn't wrk with Puppeteer

I try to scroll area inside div using Puppeteer. I tried to follow these answers: https://stackoverflow.com/a/67490337 and https://stackoverflow.com/a/52031392

How do I make puppeteer select google translate text box for me? it only works if I click it myself

my code is as follows await page.waitForTimeout(1000); await page.waitForSelector( "#yDmH0d > c-wiz > div > div.WFnNle > c-wiz > div.OlSOob

`puppeteer-page-proxy` with `https` has Error: net::ERR_FAILED or RFC 6066 error

I use puppeteer and puppeteer-page-proxy proxy. I don't have problem with use args: ['--proxy-server=socks5://127.0.0.1:9150',... with either http and https. I

How to run latest chromium/puppeteer on firebase

I do have a working application on firebase, that renders code and delivers a png image as output using puppeteer. However I would like to have a newer chromium

How to print the raw devtools request sent by Puppeteer?

I see that Puppeteer used devtools protocol. I want to see what requests are sent by Puppeteer. https://github.com/puppeteer/puppeteer const puppeteer = require

Youtube cookies auto accept with puppeteer

Im trying to find something can automatically accept or delete youtube cookies with puppeteer when i load "youtube.com" Example of popup you can get in private

Running a for loop within a Puppeteer test

I'm trying to run a for loop within a Puppeteer test that sends values to a search bar: test('test one', () => { var results = [""] var fuzz = ['ap

getting error trying to connect proxy to pupeteer

Im getting the error Error: net::ERR_TIMED_OUT at http://lumtest.com/myip.json const puppeteer = require('puppeteer'); (async () => { const browser = aw

Problem scraping Bet365 with headless mode

A few days ago I started the development of a bot to capture data/results from virtual sports (specifically football) at Bet365 (note: I know this is not allowe

Browser crashes on auto-scroll puppeteer

I have a scraper that scrapes the video urls of all the posts of a user. I have implemented auto-scroll so that all the videos are loaded for me to scrape. I te

Puppeteer getting response from url pdf

I'. automating regression tests for a website and one of the tasks is to check the various urls that come in to me. I am using Puppeteer and Chromium for this.

node js puppeteer How do I scrape values ​from duplicate selector?

I want to scrape the value, but its selector is duplicate, I don't know how to solve. It will always scrape the value of the one above. here my code: const pupp

How to select and click a child div with Puppeteer

I am new to Puppeteer; trying to click on the first div after div with class version-cell create-update. Not all web pages I scrape will have the version-cell c

puppeteer bypass cloudflare by enable cookies and Javascript

(In nodeJs -> server side only). I'm doing some webscraping and some pages are protected by the cloudflare anti-ddos page. I'm trying to bypasse this page. B

How do I access variables outside a scope inside it?

So here's my code, I'm trying to access the anum and articleHTML inside the page.evaulate scope. It says anum and articleHTML is not defined const num = context

TimeoutError: Timeout exceeded while waiting for event after click on button in autotest

Registration on the site consists of three stages. The first step is to provide a name, email address. When you submit the form of the first step, you are redir

Bypass Cloudflare with puppeteer

I am trying to scrape some startups data of a site with puppeteer and when I try to navigate to the next page the cloudflare waiting screen comes in and disrupt

Typescript auto-complete not working in VSCode

I'm trying to make use of Puppeteer within a ts file which works fine except the VSCode Intellisense stops working a soon as I'm inside a .ts and not a .js file

Unable to login to Youtube acc through google sign-in with puppeteer

When executing my puppeteer code, I'm unable to get past the email page after clicking next because of some sort of bot protection from Google stating "Try usin