'grep -v with newlines not behaving as expected

I'm trying to create a pre-commit trigger for git, I have to use bin/sh for maximum compatibility so please keep to what will work with sh (not bash etc)

I'm not a unix developer, so there is probably something pretty fundamental I'm not grasping here, but I can't seem to discover it.

I have a list of the files in a variable. I want to remove those with certain suffixes.

what I thought would work, does work here: https://www.online-utility.org/text/grep.jsp

Input Regex: ^.+(\.auto\.sql|\.sln)$

src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln
src/blah/Some Other File.sql

Invert Match (Display Non-Matching Lines)

Correctly returns

src/blah/Some Other File.sql

But when I put it into a sh script it doesn't work (I'm using https://www.jdoodle.com/test-bash-shell-script-online/)

#!/bin/sh
files="src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln
src/blah/Some Other File.sql"

printf "%s" "$files"
numfiles=$( printf '%s' "$files" | grep -c '$' )

printf "\n%s\n" $numfiles

#files=$( printf '%s' "$files" | grep -v "\.auto\.sql") # works but diesn't guarantee end of line
#printf "%s" "$files"

files=$( printf '%s' "$files" | grep -v "^.+(\.auto\.sql|\.sln)$") # doesn't work even though it should match
printf "%s" "$files"

returns

src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln
src/blah/Some Other File.sql
4
src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln
src/blah/Some Other File.sql
  • matching with non-end of line tokens works but the end of line doesn't.
  • However -E works fine and finds only the rows I don't want
src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln
src/blah/Some Other File.sql
4
src/blah/auto/star.LOAD_DimBrandDataLevels.auto.sql
src/blah/auto/star.LOAD_DimBrandDataLevels_FROM_Mds_mdm_BrandDataConfidence.auto.sql
src/blah/Blah.sln

Really not sure what is going on and have exhausted several avenues.

Hopefully someone can shine a bit of light on how to solve this. Thanks

sh


Solution 1:[1]

You need to use -E, --extended-regexp:

$ printf "%s" "$files" | grep -v -E "^.+(\.auto\.sql|\.sln)$"
src/blah/Some Other File.sql

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Necklondon

Related Questions