'How to get content inside tag with beautifulsoup

I am trying to get the number of comments that a video has received on youtube with requests and beautifulSoup. When I checked the elements tab under developer options in chrome this is what I found

To target the yt-formatted-string tag I did the following

response = requests.get(url)
soup = bs(response.text, "html.parser")

print(soup.find("yt-formatted-string", {"class": "count-text"}).text)

But this gives me the following error

AttributeError: 'NoneType' object has no attribute 'text'

I guess this happens as the tag was not found. So I tried printing all the tags using soup.find_all() as follows

print([tag.name for tag in soup.find_all()])

This gives the following result

['html', 'head', 'meta', 'script', 'script', 'script', 'link', 'link', 'link', 'link', 'link', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'link', 'script', 'link', 'link', 'link', 'link', 'style', 'style', 'style', 'style', 'style', 'meta', 'link', 'link', 'script', 'script', 'link', 'link', 'link', 'title', 'meta', 'meta', 'meta', 'link', 'link', 'link', 'link', 'link', 'link', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'div', 'link', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'span', 'link', 'link', 'script', 'link', 'span', 'link', 'meta', 'meta', 'link', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'meta', 'body', 'script', 'div', 'div', 'div', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'iframe', 'ytd-app', 'ytd-masthead', 'div', 'div', 'input', 'svg', 'g', 'path', 'div', 'a', 'svg', 'g', 'g', 'path', 'path', 'g', 'g', 'path', 'path', 'path', 'path', 'path', 'path', 'path', 'a', 'svg', 'g', 'g', 'path', 'path', 'g', 'path', 'path', 'path', 'path', 'path', 'path', 'path', 'path', 'span', 'div', 'div', 'div', 'div', 'div', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a', 'div', 'div', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'div', 'script', 'script', 'script', 'script', 'script', 'script', 'script', 'script']

I notice that the tag yt-formatted-string does not exist in the list.

How do I get the comments count using beautifulsoup?

PS: I can achieve this using Selenium, but there are other requirements which make selenium not so desirable here.

Solution 1:^[1]

If you inspect the HTML you'll see the following line in the <head>:

<meta itemprop="interactionCount" content="75068">

This indicates the viewcount and is much more easy to find using BeautifulSoup

import requests
from bs4 import BeautifulSoup

pages = requests.get('https://www.youtube.com/watch?v=HssXq2OsumY')
soup = BeautifulSoup(pages.content, 'html.parser')

interactionCount = soup.find('meta', attrs={'itemprop':'interactionCount'})['content']

print(interactionCount)

Will show 75077 for this youtube vid

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	0stone0

'How to get content inside tag with beautifulsoup

Solution 1:[1]

Sources

Related Questions

Solution 1:^[1]