'download html web page or scrape text from it
I am struggling to download full html code from below page. I was using URLDownloadToFileA and MSXML2.XMLHTTP60 methods and none of them downloads full code. Part of the webpage is missing - table on the bottom with "DEBTI...." in not included in the code. My aim is to get these "DEBTI..." strings and I want to avoid selenium if possible. None of below codes work for me, but when I just ctrl+s (save) page from the browser everything is correctly mentioned in the code. Any suggestions? Thanks!
Sub Get_Data()
Dim xmlhttp As New MSXML2.XMLHTTP60, myurl As String
myurl = "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=2309&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=Thumbnail&isVisitedRef=false&allRecords=0&showProgressBar=true" 'replace with your URL
xmlhttp.Open "GET", myurl, False
xmlhttp.Send
Open "C:\Desktop\test.txt" For Output As #1
Print #1, (xmlhttp.responseText)
Close #1
End Sub
Private Declare PtrSafe Function URLDownloadToFile Lib "urlmon" _
Alias "URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, _
ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Function DownloadFile(URL As String, LocalFilename As String) As Boolean
Dim lngRetVal As Long
lngRetVal = URLDownloadToFile(0, URL, LocalFilename, 0, 0)
If lngRetVal = 0 Then DownloadFile = True
End Function
Sub test()
DownloadFile "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=8418&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=List&isVisitedRef=false&allRecords=0&showProgressBar=true", "C:\Desktop\abc.html"
End Sub
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
