'How to wait for a page finish render and get complated page source

I used Selenium(version 4.1.0) in a C# project.I looked up a lot of information. This is My code

ChromeDriver driver = new ChromeDriver();
driver.Navigate().GoToUrl("xxxx");
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(30));
wait.Until(ExpectedConditions.ElementExists(By.Id("xxx")));
context.Response.Write(driver.pageSource);

This is very effective for get full page html until I test https://jp.mercari.com. It can not get full page.

Is there any other way to get the complated page source?

c++sharpasp.netselenium


Solution 1:[1]

You aren't getting the get full page html as a lot of the elements are distributed within multiple #shadow-root (open) elements.

mercari

Conclusion

Even after the webpage compltes rendering the HTML DOM, the entire pageSource may not be available. To access the entire DOM Tree you have to use Document.querySelector().

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1

Related Questions