'Extract part of the code and parse HTML in bash

I have external HTML site and I need to extract data from the table on that site. However source of the HTML website has wrong formatting except the table in the code, so I can not use 

xmllint --html --xpath <xpath> <file>

because it does not work properly, when HTML formatting on the site is broken.

My idea was to use curl and delete code above and below the table. When table is extracted, code is clean and it fits to xmllint tool (I can use xpath then). However delete everything above the match is challenging for shell as you can see here: Sed doesn't backtrack: once it's processed a line, it's done. Is there a way how to extract only the code of the table from the HTML site in bash? Suposse, code has this structure.

<html>
<head>
</head>
<body>
<p>Lorem ipsum ....</p>
  <table class="my-table">
    <tr>
      <th>Company</th>
      <th>Contact</th>
    </tr>
  </table>
<p>... dolor.</p>
</body>
</html>

And I need output like this to parse data properly:

  <table class="my-table">
    <tr>
      <th>Company</th>
      <th>Contact</th>
    </tr>
  </table>

Please, do not give me minus because of trying to use bash.

bashsedhtml-parsing


Solution 1:[1]

For your purposes a quick solution would be a 1-liner:

sed -n '/<table class="my-table">/,/<\/table>/p'  <file>

Explanation: print everything between two specified tags, in this case <table>

You could also easily make a tag variable for e.g <body> or <p> and change the output on the fly. But the above solution gives what you asked for without external tools.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 miken32

Related Questions