'Web scraping Jetbrains webpage through python

I want to grab data from my Jetbrains admin account for license management. My code is as below.

import requests
import pyotp
from lxml import html
from bs4 import BeautifulSoup as soup


USERNAME = "username"
PASSWORD = "password"
APP = "Jetbrains IntelliJ"

LOGIN_URL = "https://account.jetbrains.com/login"
URL = "https://account.jetbrains.com/organization/254678"
mfa = pyotp.TOTP('S8I9ASC67C4H9HQE')
token = mfa.now()
#print(token)


def main():
    session_requests = requests.session()

    # Create payload
    payload = {
        "username": USERNAME, 
        "password": PASSWORD, 
        "secondFactor": token
    }

    # Perform login
    result = session_requests.post(LOGIN_URL, data = payload, headers = dict(referer = LOGIN_URL))

    # Scrape url
    result = session_requests.get(URL, headers = dict(referer = URL))
    print(result.text)

if __name__ == '__main__':
    main()

When I print the token, there is no errors. But when I try to print the result, still no errors, I cannot log into the admin console. It will show the login page.

The result is:

<!DOCTYPE html>

<html>
<head>
<title>JetBrains Account</title>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1.0" name="viewport"/>
<meta content="origin-when-cross-origin" name="referrer"/>
<link href="https://resources.jetbrains.com/storage/ui/fonts/fonts.css" rel="stylesheet" type="text/css"/>
<link href="/appresources/jquery.dataTables.css?v=c2ef715" rel="stylesheet" type="text/css"/>
<link href="/appresources/markside.css?v=a15765bc" rel="stylesheet" type="text/css"/>
<link href="/appresources/bootstrap-full.less?v=ef882a8b" rel="stylesheet" type="text/css"/>
<link href="/appresources/datetimepicker.css?v=5df2ea75" rel="stylesheet" type="text/css"/>
<link href="/appresources/jasny-bootstrap.css?v=f55b5bde" rel="stylesheet" type="text/css"/>
<link href="/appresources/font-awesome.css?v=de015a3" rel="stylesheet" type="text/css"/>
<link href="/appresources/tipsy.css?v=980f2268" rel="stylesheet" type="text/css"/>
<link href="/appresources/chosen.css?v=82ebffd4" rel="stylesheet" type="text/css"/>
<link href="/appresources/credit-card.less?v=9fede289" rel="stylesheet" type="text/css"/>
<link href="/appresources/jquery.growl.css?v=26245b98" rel="stylesheet" type="text/css"/>
<link href="/appresources/bootstrap-multiselect.css?v=49de55a1" rel="stylesheet" type="text/css"/>
<link href="/dynamic/default.css?v=a4af43ce" rel="stylesheet" type="text/css"/>
<link href="/static/favicon.ico" rel="shortcut icon" type="image/x-icon"/>
<script id="ze-snippet" src="https://static.zdassets.com/ekr/snippet.js?key=06c7e4c3-a30c-4343-bdc7-7c95ac2a6110" type="text/javascript"></script>
<link href="https://account.jetbrains.com/login" rel="canonical"/><meta class="local" content="JetBrains, creator of the best Java IDE - IntelliJ IDEA - is a technology-leading software vendor specializing in the creation of intelligent development tools." name="description"/> </head>
<body class="jba-agreement" onunload=";">
<div class="jba-agreement-panel">
<div class="container">
<div class="row">
<div class="col-sm-12">By using the JetBrains Account website, you agree to the JetBrains Account Agreement.<span class="unit-container-sm"></span><a href="/profile/privacyPolicy?backUrl=https%3A%2F%2Faccount.jetbrains.com%2Flogin">Review now</a><span class="unit-container-sm"></span><a class="ajax-action" data-method="POST" data-url="/profile/postponePrivacyPolicy?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" href="#">Remind me later</a></div>
</div>
</div>
</div>
<div class="navbar navbar-default navbar-fixed-top navbar-inverse">
<div class="container">
<div class="navbar-header"><button class="navbar-toggle" data-target=".navbar-menu-collapse" data-toggle="collapse" type="button"><span class="sr-only">Toggle navigation</span><i class="icon icon-bar"></i><i class="icon icon-bar"></i><i class="icon icon-bar"></i></button><a class="logo" href="https://www.jetbrains.com">JetBrains</a></div>
<div class="collapse navbar-collapse navbar-menu-collapse">
<ul class="nav navbar-nav"></ul>
<ul class="nav navbar-nav navbar-actions">
<li><a href="https://sales.jetbrains.com/hc/categories/200934669-JetBrains-Account-Help"><span class="j-icon-help-medium"></span>Help</a></li>
</ul>
</div>
</div>
</div>
<div class="footer-wrap">
<div class="header"></div>
<div class="container">
<div class="js-concurrent-resource-modification-alert" style="display:none;">
<div class="alert alert-danger"><a aria-label="Close" class="close">×</a>Please <a class="js-reload-page-link">reload current page</a> because it's content or state has been changed outside.</div>
</div>
<div class="row">
<div class="col-sm-12">
<div class="container start-screen">
<div class="row">
<div class="col-sm-12 col-md-11 col-lg-10">
<div class="row">
<div class="col-sm-6 col-lg-5">
<h1>Welcome to JetBrains Account</h1>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-purchases-feature pull-left"></i>
<p class="lead">
<strong>Access your purchases</strong>
<br/>
                             and view your order history
                          </p>
</div>
</div>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-alert-feature pull-left"></i>
<p class="lead">
<strong>Identify expired and outdated licenses,</strong>
<br/>
                             order new licenses and upgrades
                          </p>
</div>
</div>
<div class="row">
<div class="col-sm-12">
<i class="j-icon-manage-feature pull-left"></i>
<p class="lead">
<strong>Manage your company licenses</strong>
<br/>
                             and distribute them to end users
                          </p>
</div>
</div>
</div>
<div class="col-sm-4 col-sm-offset-1">
<div class="well panel-highlighted">
<form action="/authorize?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" class="js-auth-dialog-form" method="post">
<p class="js-auth-dialog-div-errors text-danger"></p>
<h5>Sign in with existing account</h5>
<div class="form-group js-auth-dialog-input-username"><input autofocus="" class="form-control" id="username" name="username" placeholder="Email address or Username" style="width:100%" type="text"/></div>
<div class="form-group js-auth-dialog-input-password"><input class="form-control" name="password" placeholder="Password" style="width:100%" type="password"/></div>
<div class="form-group js-auth-dialog-input-2fa" style="display:none;">
<p class="small">You have enabled two-factor authentication, so you will also need to enter the 6‑digit verification code from your mobile authenticator app, or use a recovery code.</p>
<input class="form-control" name="secondFactor" placeholder="One-time password" style="width:100%" type="text"/>
</div>
<div class="form-group clearfix">
<div class="pull-left"><button class="btn btn-primary btn-lg js-auth-dialog-btn-signin login-submit-btn" onclick="return false;">Sign In</button><span class="unit-container-xs"></span></div>
<div class="js-auth-dialog-div-reset-pswd"><a class="btn btn-lg btn-link" href="/forgot-password">Forgot password?</a></div>
</div>
</form>
</div>
<div class="well panel-highlighted">
<form action="/signup-request?_st=sWzaeWLBFTOdAHuj1ArjCu3TsF-9KcNHdRBXfmco7qRQ9QN7K_H7x-cKILudA3A9" method="post">
<h5>Not registered yet?</h5>
<h3>Create JetBrains Account</h3>
<div class="form-group"><input class="form-control" id="email" name="email" oninput="try { if($(this).val().trim().match(new RegExp(this.getAttribute('pattern')))) { this.setCustomValidity(''); } else { this.setCustomValidity('Please enter a valid email address'); } } catch(e) {}" pattern="^[a-zA-Z0-9!#$%&amp;'*+/=?^_`{|}~-]+(?:\.[a-zA-Z0-9!#$%&amp;'*+/=?^_`{|}~-]+)*@(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])?\.)+[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]$" placeholder="Your email address" style="width:100%" type="text"/></div>
<div class="form-group clearfix"><button class="btn btn-primary btn-lg eml-submit-btn" onclick="$('#email').val(function(_, x) { return x.trim() })">Sign Up</button></div>
</form>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="footer-push"></div>
</div>
<div class="footer">
<div class="container">
<div class="small">Copyright © 2000–2020 JetBrains s.r.o. | <a href="https://sales.jetbrains.com/hc/categories/200934669-JetBrains-Account-Help" target="_blank">Help</a> | <a href="mailto:[email protected]?subject=JetBrains%20Account%20Support">Support</a> | <a href="https://www.jetbrains.com/company/privacy.html" target="_blank">JetBrains Privacy Policy</a> | <a href="//www.jetbrains.com/legal/agreements/jetbrains_account.html">JetBrains Account Agreement</a> | Build #2020.08-2134</div>
</div>
</div>
<script src="/appresources/requirejs-config.js?v=d2d44ead" type="text/javascript"></script>
<script src="/appresources/require.js?v=cc5cbeb6" type="text/javascript"></script>
<script type="text/javascript">
    requirejs.onError = function(error) {
        require(['jetprofile'], function (JetProfile) {
            console.error(error.message, error.stack || error.originalError.target.src);
            window.onerror(error.message, error.stack || error.originalError.target.src);
        });
    };
    require(['/appresources/main.js']);</script>
</body>
</html>

Is there any method to be changed to get the access?

Solution 1:^[1]

Emailed their support as this is something I've been trying to do as well. They've created an API: https://sales.jetbrains.com/hc/en-gb/articles/4406854152978

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	sqpc

'Web scraping Jetbrains webpage through python

Solution 1:[1]

Sources

Related Questions

Solution 1:^[1]