I am doing some experiments with Python3.6 in Mac and BeautifulSoup. I am trying to build a simple program to scrap song lyrics from a URL and store them as pla
I am trying to streamline my data collection by using Python 3.7 and BeautifulSoup to pull company name, if that company is approved or other, and if they are m
I am trying to get a JSON response from the link used as a parameter to the urllib request. but it gives me an error that it can't contain control characters. h
OpenSea allows users to buy and sell NFTs. From OpenSea, you can view the prices of listed NFTs within a project. When an NFT is listed, is the listed price sto
I'm trying to scrape the source of the first image with a specific class. On the page, there are multiple images with different additional classes but they shar
So I have set up a program that the goal of which is to run through every possible ID and test that ID to see if there is a discord emoji URL associated with it
I'm using BeautifulSoup to scrape text from a website, but I only want the <p> tags for organization. However, I can't use text.findAll('p'), because the
I'm currently trying to get COVID-19 from the Covid Data Repository by Johns Hopkins. https://github.com/CSSEGISandData/COVID-19 The repo get updated with new d
I'm using selenium-wire with undetectable chromedriver and it's giving me: "Your Connection To This Site Is Not Secure" when I go into a site, and the https in
I am trying to scrape a data from opensea. I can scroll page but I dont know how could collect data while scrolling. my code from selenium import webdriver fro
I need to find the phone numbers in this website, I have come to the conclusion that I need to write an If statement but I'm not really sure how to do that sinc
import os import selenium from selenium import webdriver import time browser = webdriver.Chrome() browser.get('https://www.skysports.com/champions-league-fixtu
I am scraping a table and each row has a button to show a modal with information. I need to scraping the information from the modal for each row but I dont know
I want to scrape 2015 from below HTML: I use the below code but am only able to scrape "Annee" soup.find('span', {'class':'optionLabel'}).get_text() Can someo
i'm triying to fix a data crawler that until last couple of weeks was working perfectly. The script consist of two parts, one that retrieves the links of the ar
I am learning nodejs/puppeteer and having issues getting Puppeteer to fill UPC numbers from a CSV file onto the search bar of a book website. I managed to get a
Using puppeteer, I have a specific page that I am web-scraping for data and screenshot-ing for proof that the data is correct. The web page itself includes a bu
I started playing around with Twitter API v2 in Tweepy. I've had some experience with v1 but it looks like it's changed a bit. I'm trying to search tweets based
newbie here. I was on web scraping project. And I wanted some guide on web scraping pagination technique. I'm scraping this site https://www.imoney.my/unit-trus
i've created function using selenium undetected chromedriver in order to create a google chat with email specifeid. And every time i run my code i have to log i