How can I get the link of all the posts in the Instagram profile with Selenium?

Question

I'm trying to get the links of all the posts in an instagram profile. How can I get to the href="/p/CX067tNhZ8i/" in the photo.

What I'm trying to do is find the href= blabla of all posts.

All your posts are in class="v1Nh3 kIKUG _bz0w".

I tried to get the hraf= blabla value from this class with the get_attribute command, but it didn't work.

Thank you for your help.

browser.get("https://www.instagram.com/lightning.mcqueen34/")

links = []
elements = browser.find_element_by_xpath('//*[@id="react-root"]/div/div/section/main/div/div[4]/article/div[1]/div/div[1]/div[3]')
for i in elements:
    links.append(i.get_attribute('href'))

I thought this would work but the elements value is not a list . It gave an error.

Answer 1

This should work:elements = browser.find_elements_by_tag_name('a')

Below answer will not work in all cases, dependant on how the DOM of the page is loaded.

Replace this line:

elements = browser.find_element_by_xpath('//*[@id="react-root"]/div/div/section/main/div/div[4]/article/div[1]/div/div[1]/div[3]')

With:

elements = browser.find_element_by_xpath("//a[@href]")

This will let you retreive all links with a href from the page.

Answer 2

Try to change XPath first to get the DIV class or ID after trying this //a[@href] Xpath to get all HREF.