Category "beautifulsoup"

Extracting contents of html into BeautifulSoup if search for script ID is a success

My html is below: <html> <body> <div> ... </div> <script id="1" ...> </script> </body> </ht

How to scrape data from Twitter without its API using BeatifulSoup

I'm currently trying to scrape some data from Twitter, like username, screen name, the content of the tweet etc. But I've run into some problems: I've been tryi

find_all() prints everythigh twice

I just started my first Web scraping project and out of some reason when I try to run this simple code, it prints all of the headlines twice. I have no Idea why

How to scrape related searches on google?

I'm trying to scrape google for related searches when given a list of keywords, and then output these related searches into a csv file. My problem is getting be

BS4 - 'NoneType' object has no attribute 'findAll' when scanning spans on amazon page

I'm following a Udemy course on learning BS4 and it seems to be a bit outdated so I'm having trouble with this part. The objective is to scrape the price of thi

Is there any way instead of status_code to determine the request is true or false?

I'm using Python3 with BeautifulSoup. I want to scrape data for a few employees from a site, depending on their ID number. My code: for UID in range(201810000,2

Can't parse out text that is behind </span>text</a> in Beautifulsoup

I think I have tried it all, read crummy, read documentation on Beautifulsoup4 website. I can't get this thing wrapped around my head. So to the question: &

How to select and scrape specific texts out of a bunch <ul> and <li>?

I need to scrape "2015" and "09/09/2015" from the below link: lacentrale.fr/auto-occasion-annonce-87102353714.html But since there are many li and ul, I cant sc

How to scrape sofifa website positions. Text inside of span beautiful soup

So I am webs scraping the sofifa website into a workable csv. Each player gets a column. My main problem is the position section of the website is only exportin

Problem in fetching long URLs using BeautifulSoup

I am trying to fetch a URL from a webpage, here is how the URL looks in the Inspect section: Here is how the URL looks in my python-code: How can I get the ac

Why is web scraping stock prices through beautiful soup returning a different price than the one on the Yahoo Finance page?

I am trying to write a program that will give me the stock price for a few different stocks, but when I run my program, it returns 116.71, while Yahoo Finance h

Python terminal closes when importing BeautifulSoup

I have a simple python program, that is supposed to scrape some information from the internet and do stuff with it. When I run the code in PyCharm (IDE) it work

Why I'm getting "UnicodeEncodeError: 'charmap' codec can't encode character '\u25b2' in position 84811: character maps to <undefined>" error?

I'm getting UnicodeEncodeError: 'charmap' codec can't encode character '\u200b' in position 756: character maps to error while running this code:: from bs4 imp

How to ignore infobox when scraping title from Wikipedia anchor text?

I am trying to scrape the first 20 links on a Wikipedia page but I want to ignore the infobox on the right side. It has a 'table' tag. Here is what I have so fa

Webscraping Google Search Results Using Google API - Returns same result over and over again

My problem Hi everyone I am attempting to develop my very first web scraper using the Google API and Beautiful Soup in Python. The aim is for the scraper to

Scraping google play reviews

I am new to programming and I have recently tried to scrape google play reviews with python using the following program: from bs4 import BeautifulSoup import u

Transfermarkt webscrapping failed

I have created the code below to scrape data from Transfermarkt but it gives error ERROR IndexError Traceback (most recent call l

Transfermarkt webscrapping failed

I have created the code below to scrape data from Transfermarkt but it gives error ERROR IndexError Traceback (most recent call l

Not able to fetch <h3> ag from the below website using Beautiful Soup

I'm trying to fetch top 100 movie names, but not able to access h3 tag.How can I fetch it from this link? Edit - Using below code to extract h3 - import request

How is data scraping based on location in Amazon?

Whenever I want to scraping on amazon.com, I fail. Because Product information changes according to location in amazon.com This changing information is as follo