Parsing a table data from BSE site into Python

Question

I am new to python. I want to parse a data from a table in BSE site into python. I tried using beautifulsoup module but I am unable to know which reference to use, so as to find the correct table. In fact even that particular table row is not getting displayed in python

The code that I tried was:

from urllib.request import Request, urlopen       
from bs4 import BeautifulSoup as soup    
page = 'https://www.bseindia.com/stock-share-price/itc-ltd/itc/500875/corp-actions/'    
req = Request(page, headers = {'User-Agent': 'Mozilla/5.0'})    
webpage = urlopen(req).read()    
page_soup = soup(webpage, "html.parser")    
containers = page_soup.findAll("table", id = "tblinsidertrd")

This is giving a blank [ ] result.

Then I tried

containers = page_soup.findAll('td')    
containers = page_soup.findAll('tr)

In both results I was unable to find the table or data I was looking for. I couldn't even find the table headings viz 'EX Date' and 'Amount'

The table that I want from BSE site is highlighted below:

Please help me as to where I am going wrong and why I am unable to view the dividend table data?

Answer 1

The content is dynamically generated. You can pull it form the api:

import pandas as pd
import requests

url = 'https://api.bseindia.com/BseIndiaAPI/api/CorporateAction/w?scripcode=500875'
headers = {'User-Agent': 'Mozilla/5.0'}
jsonData = requests.get(url, headers=headers).json()

df = pd.DataFrame(jsonData['Table'])

Output:

print(df)
    Amount    BCRD_from      purpose_name
0    10.15  06 Jul 2020          Dividend
1     5.75  22 May 2019          Dividend
2     5.15  25 May 2018          Dividend
3     4.75  05 Jun 2017          Dividend
4     8.50  30 May 2016          Dividend
5     6.25  03 Jun 2015          Dividend
6     6.00  03 Jun 2014          Dividend
7     5.25  31 May 2013          Dividend
8     4.50  11 Jun 2012          Dividend
9     2.80  10 Jun 2011          Dividend
10    1.65  10 Jun 2011  Special Dividend
11   10.00  09 Jun 2010          Dividend
12    3.70  13 Jul 2009          Dividend
13    3.50  16 Jul 2008          Dividend
14    3.10  16 Jul 2007          Dividend
15   10.00  03 Jul 2001          Dividend