'Improve performance of shell script used to update HTML files

I am using the below code to generate HTML files. There are several hundred HTMLs that get updated & each have several thousand URLs in them that gets updated.

I thought the below code would be reasonably fast to execute. However, it's taking over an hour to process the data which is unacceptable. Sometimes it can be over 2 hours.

Please can I get help to improve the performance of this code?

update_index_page(){
   echo "Updating $1"
        mega_string=''
        while read line
        do
                subitem=$(echo "${line}" | awk -F"#" '{print $3}')
                short_url="<a href=\"https://mywebsite.com/${1}/${subitem}.html\" target=\"_blank\" rel=\"noopener\">${subitem}</a>"
                mega_string="${mega_string}<tr><td>${short_url}</td><td>$(echo "${line}" | awk -F"#" '{print $2}')</td><td>$(echo "${line}" | awk -F"#" '{print $1}')</td></tr>"
        done<"${TMP_LOC}/${1}.txt"
        echo "${HTML_PART_1}${mega_string}${HTML_PART_2}${HTML_PART_3}" > ${CODE_LOC}${1}/index.html
        if [[ "${1}" == "All-Items" ]]
        then
                to_replace=$1"andALL"
                sed -i "s/PLACEHOLDEREXCHANGENAME/$to_replace/g" ${CODE_LOC}${1}/index.html
        else
                sed -i "s/PLACEHOLDEREXCHANGENAME/$1/g" ${CODE_LOC}${1}/index.html
        fi
}


while read line
do
    update_index_page "$line"
done<"$INPUT_FILE"
bashshell


Solution 1:[1]

I re-wrote the section in python which takes about 10 min to complete as against 2 or more hours earlier.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 usert4jju7

Related Questions