How to get the url from extracted information from a website

Question

So basically I am stuck on the problem where I don't know how to the url from the extracted data from a website.

Here is my code:

import requests
from bs4 import BeautifulSoup

req = requests.get('https://api.randomtube.xyz/video.get?chan=2ch.hk&board=b&page=1')

soup = BeautifulSoup(req.content, "html.parser")

print(soup.prettify())

I get a lot of information on output, but the only thing I need is the url, I hope someone can help me.

P.S:

It gives me this information:

{"response":{"items":[{"url":"https:\/\/2ch.hk\/b\/src\/262671212\/16440825183970.webm","type":"video\/webm","filesize":"20259","width":1280,"height":720,"name":"1521967932778.webm","board":"b","thread":"262671212"},{"url":"https:\/\/2ch.hk\/b\/src\/261549765\/16424501976450.webm","type":"video\/webm","filesize":"12055","width":1280,"height":720,"name":"1526793203110.webm","board":"b","thread":"261549765"}...

But i only need this part out of all the things https:\/\/2ch.hk\/b\/src\/261549765\/16424501976450.webm (Not exactly this url, but just as an example)

Answer 1

The problem is you are telling BeautifulSoup to parse JSON data as HTML. You can get the URL you need more directly with the following code

import json
import requests
from bs4 import BeautifulSoup

req = requests.get('https://api.randomtube.xyz/video.get?chan=2ch.hk&board=b&page=1')

data = json.loads(req.content)
my_url = data['response']['items'][0]['url']

Answer 2

You can do it this way:

url_array = []

for item in soup['response']['items']:
  url_array.append(item['url'])

I guess if the API returns JSON data then it should be better to just parse it directly.