'Chrome devtools returns null when element is clearly on page

I try to implement a webscraper.

I thought the issue was my rust code for the longest time but as the title suggests it seems to be the query selector I am trying to use to locate the element. I am trying to crall this page and extract the href on the twitter follow button.

When using $$("#follow-button") in chrome devTools I get a null response until the element is inspected then the query returns the correct thing. Could anyone please shed some light on why this element doesn't seem to exist until inspected? I can provide links to rust code if that would be helpful.



Solution 1:[1]

The link is in an iframe. You have to select the Iframes src, and request for it's document, then you can select any of its children.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Invizi