'I'm using Beautiful Soup to web scrape a website with a password, but it's acting funky

Like I said in the title, I'm trying to web scrape a website that's behind a password. This is the code I have so far:

import requests
from bs4 import BeautifulSoup as bs

url = ('https://www.dummyurl.com')
login = ('/website-extension/')

headers = {'User-Agent': 'User agent information', 'origin': url, 'referer': url + login}

s = requests.session()

login_payload = {
    'clientcode': 'uhia8',  #This is not the actual info for obvious reasons
    'username': 'jferguson',
    'password': 'qwerty123'
}

login_req = s.post(url + login, headers=headers, data=login_payload) 

print(login_req)

I got <Response [200]>, which from my understanding means that I logged in successfully, but when I, in the session, tried to print the html for the page that I logged into like in the code below, it prints the login page HTML:

soup = bs(s.get(url + login).text, 'html.parser')

print(soup.prettify)

Any idea on what I'm doing wrong? I'm relatively new to coding, and I'm new to stackoverflow, so please be nice :)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source