'trouble logging in to LinkedIn with requests_html

I'm trying to scrape data from linkedin. to do that I'll need to login, the issue is that I get an error in the html tag. here is the code.

url = 'https://www.linkedin.com/checkpoint/lg/login-submit'
s = AsyncHTMLSession()
headers = {
    'User_Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36'
}
payload = {
    'session_key': key,
    'session_password': password
}
r = await s.post(url, data = payload, headers = headers)
print(r.html)

traceback:

<html url='https://www.linkedin.com/checkpoint/lg/login?errorKey=unexpected_error'>

EDIT:

I realized that it was probably a rendered page or some kind of trickery, so I used selenium. can't be stubborn with code.

python screen-scraping

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'trouble logging in to LinkedIn with requests_html

Sources

Related Questions