'Python post request with wrong form [no "action", "method", or "value attribute"]

Want

Make Python submit an address on https://boligejer.dk/ and collect the response.

Specifically, I want to get the response from submitting "Ved Stranden 1, 3220 Tisvildeleje" - shown in first picture. And then collect the response (https://boligejer.dk/ejendomsdata?knr=0270&enr=027002) - shown in second picture

Submitting an address

Response page after submitting the address

Have Followed recipe from: Python post request for USPTO site scraping. However, I am not getting my form correct. Any suggestions or input on how to nail this?

from bs4 import BeautifulSoup
import requests
import re

#Methods
def make_soup(url):
r = requests.get(url)
soup = BeautifulSoup(r.text, 'lxml')
return soup

def get_forms(soup):
    form = soup.find_all("form")
    return form

def get_form_data(form,address): 
    data = {}
    for inp in form('input'):
     # if the value is None, we put the address to this field
     data[inp['name']] = inp['value'] or address

return data 

#Values
url               = 'https://boligejer.dk/'
form_name         = 'content-banner__form mt-3'
address_wanted    = "Ved Stranden 1, 3220 Tisvildeleje"

soup               = make_soup(url)
all_forms          = get_forms(soup)
form               = all_forms[1]

#Method does not inject address, as there is no value with none
#data               = get_form_data(form,address)    
data               = {'input aria-autocomplete=':"list","aria-label":address_wanted,
                 "autocomplete":"off","class":"form-control","id":"dawa-autocomplete- 
                   input","placeholder":"Indtast adresse","type":"text"}

request_with_form    = requests.post(url, data=data)


print('_______________Form from web 
page__________________________________________________________________________')
print(form)
print('_____________inputs of the form 
______________________________________________________________________')
print(form('input'))
print('_____________Data 
______________________________________________________________________')
print(data)
print('_____________Reponse with form 
______________________________________________________________________')
BeautifulSoup(request_with_form.content, 'html.parser' )

Failed output

_______________Form from web page__________________________________________________________________________
<form class="content-banner__form mt-3">
<div class="input-group">
<div class="autocomplete-container">
<input aria-autocomplete="list" aria-label="Indtast adresse" autocomplete="off" class="form-control" id="dawa-autocomplete-input" placeholder="Indtast adresse" type="text"/><div></div>
</div>
<div class="input-group-append">
<button class="btn btn-secondary btn-lg btn-icon" id="ejdsearch" type="submit">
<span class="icon">
<svg focusable="false"><use xlink:href="/themes/custom/newerst/dist/svg/svg-sprite.svg#arrow-line-right"></use></svg>
</span>
</button>
</div>
</div>
</form>
_____________inputs of the form ______________________________________________________________________
[<input aria-autocomplete="list" aria-label="Indtast adresse" autocomplete="off" class="form-control" id="dawa-autocomplete-input" placeholder="Indtast adresse" type="text"/>]
_____________Data ______________________________________________________________________
{'input aria-autocomplete=': 'list', 'aria-label': 'Ved Stranden 1, 3220 Tisvildeleje', 'autocomplete': 'off', 'class': 'form-control', 'id': 'dawa-autocomplete-input', 'placeholder': 'Indtast adresse', 'type': 'text'}
_____________Reponse with form ______________________________________________________________________

<!DOCTYPE html>

<html class="no-js theme-boligejer" dir="ltr" lang="da">
<head>
<meta charset="utf-8"/>
<link href="https://boligejer.dk/" rel="canonical"/>
<meta content="BOLIGEJER.DK Boligejer skaber overblik, indeholder guides og tilbyder relevante værktøjer til dig, der skal købe eller sælge en bolig." name="description"/>
<meta content="width" name="MobileOptimized"/>
<meta content="true" name="HandheldFriendly"/>
<meta content="width=device-width, initial-scale=1, minimal-ui" name="viewport"/>
<meta content="frontpage" name="ERST.ContentType"/>
<meta content="3. januar 2022" name="ERST.ContentDate"/>
<meta content="FORSIDE" name="ERST.ContentLabel"/>

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source

'Python post request with wrong form [no "action", "method", or "value attribute"]

Sources

Related Questions