'download html web page or scrape text from it

I am struggling to download full html code from below page. I was using URLDownloadToFileA and MSXML2.XMLHTTP60 methods and none of them downloads full code. Part of the webpage is missing - table on the bottom with "DEBTI...." in not included in the code. My aim is to get these "DEBTI..." strings and I want to avoid selenium if possible. None of below codes work for me, but when I just ctrl+s (save) page from the browser everything is correctly mentioned in the code. Any suggestions? Thanks!

Sub Get_Data()


Dim xmlhttp As New MSXML2.XMLHTTP60, myurl As String
myurl = "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=2309&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=Thumbnail&isVisitedRef=false&allRecords=0&showProgressBar=true" 'replace with your URL
xmlhttp.Open "GET", myurl, False
xmlhttp.Send

Open "C:\Desktop\test.txt" For Output As #1
Print #1, (xmlhttp.responseText)
Close #1

End Sub
Private Declare PtrSafe Function URLDownloadToFile Lib "urlmon" _
    Alias "URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, _
    ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long

Function DownloadFile(URL As String, LocalFilename As String) As Boolean
    Dim lngRetVal As Long
    lngRetVal = URLDownloadToFile(0, URL, LocalFilename, 0, 0)
    If lngRetVal = 0 Then DownloadFile = True
End Function


Sub test()

    DownloadFile "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=8418&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=List&isVisitedRef=false&allRecords=0&showProgressBar=true", "C:\Desktop\abc.html"
    
End Sub

web: https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=8418&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=List&isVisitedRef=false&allRecords=0&showProgressBar=true



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source