Why its not fetching any 'a' tags from YouTube?

Question

I am trying to get all links of videos from a given input channel link via BeautifulSoup. I found that all 'a' tags for videos have an id of 'video-title' but the code below is not giving any output:

import requests
from bs4 import BeautifulSoup

source = requests.get('https://www.youtube.com/user/TheCraftingLab/featured').text
soup = BeautifulSoup(source, 'html.parser')

container = soup.findAll("a", {id: "video-title"})
for i in container:
    print(i)

What's wrong?

Answer 1

The page you're trying to get is probably rendered with JS. So you can use requests-html module which executes JS as a web driver and returns the entire content of the loaded page.

from requests_html import HTMLSession
from bs4 import BeautifulSoup

URL = "https://www.example.com"

with HTMLSession() as session:
    response = session.get(URL)
    response.html.render()
    soup = BeautifulSoup(response.html.html, 'html.parser')

for i in soup.findAll("a", {id: "video-title"}):
    print(i)