'How to select text from multiple elements with different html structure with xpath

I have this div and I want to ask is it possible to select "TEXT_I_NEED_X" with XPATH using only 1 XPATH command ?
The closest I can get to selecting them all is this, but it selects more than I need:
//div[@class="article-text-with-img"]/p//text() 

<div class="article-text-with-img">
    
  <p>
    <a href="#"> Text1 </a>
  </p>
  
  <p>&nbsp;</p>
  
  <p>
    TEXT_I_NEED_A
    <a href="#"> Text2 </a>
  </p>
  
  <p>
    <span>
      TEXT_I_NEED_B
      <a href="#"> Text3 </a>
    </span>
  </p>
  
  <p> 
    <span>
        <span>
            TEXT_I_NEED_C
            <a href="#"> Text4 </a>
        </span>
    </span>
  </p>
  
  <p>
    <span> 
        TEXT_I_NEED_D
    </span>
    <a href="#"> Text5 </a>
  </p>

  <p>
    <span> 
        <spam>
           TEXT_I_NEED_D
        </span>
        <a href="#"> Text5 </a>
    </span>
  </p>
  
</div>
htmlweb-scrapingxpathscrapycss-selectors


Solution 1:[1]

With a single XPath expression:
//div[@class="article-text-with-img"]//a/parent::*/text() | //div[@class="article-text-with-img"]//a/preceding-sibling::span/text()

On command line with xmllint (new lines and spaces are included in text() )

xmllint --html --xpath '//div[@class="article-text-with-img"]//a/parent::*/text() | //div[@class="article-text-with-img"]//a/preceding-sibling::span/text()' test.html 



TEXT_I_NEED_A


  TEXT_I_NEED_B
  

        TEXT_I_NEED_C
        
    
 
    TEXT_I_NEED_D



    
       TEXT_I_NEED_E

Solution 2:[2]

Example with beautifulsoup:

from bs4 import BeautifulSoup

html_doc = <YOUR HTML SNIPPET FROM THE QUESTION>

soup = BeautifulSoup(html_doc, "html.parser")

article = soup.select_one(".article-text-with-img")
for a in article.select("a"):
    a.extract()

text = [t for a in article.find_all(text=True) if (t := a.strip())]
print(text)

Prints:

['TEXT_I_NEED_A', 'TEXT_I_NEED_B', 'TEXT_I_NEED_C', 'TEXT_I_NEED_D', 'TEXT_I_NEED_D']

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 LMC
Solution 2 Andrej Kesely

Related Questions