downloading all listings from nasdaq with link to json file

Question

I have been trying to download the file with code with no success so far, hope someone can help.

The cell keeps loading for hours without results unless interrupted (browser loads in seconds)
The execution just get stuck at get request forever...

import requests
url = "https://api.nasdaq.com/api/screener/stocks?tableonly=true&limit=25&offset=0&download=true"
data = requests.get(url)

Answer 1

The nasdaq.api has measures in place to prevent web scraping. You need to set additional header information to be able to access the data as recommended here..

Please try the following code:

import pandas as pd

import requests
url = \
    'https://api.nasdaq.com/api/screener/stocks?tableonly=true&limit=25&offset=0&download=true'
headers = {'Accept-Language': 'en-US,en;q=0.9',
           'Accept-Encoding': 'gzip, deflate, br',
           'User-Agent': 'Java-http-client/'}

response = requests.get(url, headers=headers)

# get json response
json = response.json()

# extract relevant keys
df = pd.DataFrame(json['data']['rows'])
df

Output:

    symbol  name    lastsale    netchange   pctchange   volume  marketCap   country ipoyear industry    sector  url
0   A   Agilent Technologies Inc. Common Stock  $135.71 -0.70   -0.513% 1403731 40728386393.00  United States   1999    Electrical Products Capital Goods   /market-activity/stocks/a
1   AA  Alcoa Corporation Common Stock  $91.96  -3.10   -3.261% 6036063 16959331342.00      2016    Metal Fabrications  Basic Industries    /market-activity/stocks/aa
2   AAC Ares Acquisition Corporation Class A Ordinary ...   $9.78   -0.01   -0.102% 3176    1222500000.00       2021    Business Services   Finance /market-activity/stocks/aac