'Rvest cannot scrape particular div section

One of my class is doing housing price prediction and we need the data for the houses. There is no available data out there with housing attributes.

https://www.movoto.com/dallas-tx/1507-padgitt-avenue-dallas-tx-75203-402_13115754/

I did some easy scraping from Finviz before and the knowledge is limited. I am able to scrap the "Property Detail", but not the "Highlights" section

url2 <- url("https://www.movoto.com/dallas-tx/6726-kenwood-ave-dallas-tx-75214/pid_0z9pcat0nh/", "rb")

html <- read_html(url2)

close(url2)

node_key <- html_nodes(html, "div.record div.key")
node_value <- html_nodes(html, "div.record div.value")
elem_value <- html_elements(node_value,"b")

text_key <- html_text(node_key)
text_key <- gsub("[\t\r\n]", "",text_key)
text_key <- gsub("\\s{2,}", "", text_key)


text_value <- html_text(elem_value)
text_value <- gsub("[\t\r\n]", "",text_value)
text_value <- gsub("\\s{2,}", "", text_value)

It didn't work for div. and xpath. I used SelectorGadget to get the xpath.

html_nodes(html, "div.mvt-attr")

html_nodes(html, xpath = "//*[(@id = "propertyHighlightPanel")]"  

enter image description here



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source