How to loop over files in directory and change path and add suffix to filename

Question

I need to write a script that starts my program with different arguments, but I'm new to Bash. I start my program with:

./MyProgram.exe Data/data1.txt [Logs/data1_Log.txt].

Here is the pseudocode for what I want to do:

for each filename in /Data do
  for int i = 0, i = 3, i++
    ./MyProgram.exe Data/filename.txt Logs/filename_Log{i}.txt
  end for
end for

So I'm really puzzled how to create second argument from the first one, so it looks like dataABCD_Log1.txt and start my program.

Answer 1

A couple of notes first: when you use Data/data1.txt as an argument, should it really be /Data/data1.txt (with a leading slash)? Also, should the outer loop scan only for .txt files, or all files in /Data? Here's an answer, assuming /Data/data1.txt and .txt files only:

#!/bin/bash
for filename in /Data/*.txt; do
    for ((i=0; i<=3; i++)); do
        ./MyProgram.exe "$filename" "Logs/$(basename "$filename" .txt)_Log$i.txt"
    done
done

Notes:

• /Data/*.txt expands to the paths of the text files in /Data (including the /Data/ part)
• $( ... ) runs a shell command and inserts its output at that point in the command line
• basename somepath .txt outputs the base part of somepath, with .txt removed from the end (e.g. /Data/file.txt -> file)

If you needed to run MyProgram with Data/file.txt instead of /Data/file.txt, use "${filename#/}" to remove the leading slash. On the other hand, if it's really Data not /Data you want to scan, just use for filename in Data/*.txt.

Answer 2

Sorry for necromancing the thread, but whenever you iterate over files by globbing, it's good practice to avoid the corner case where the glob does not match (which makes the loop variable expand to the (un-matching) glob pattern string itself).

For example:

for filename in Data/*.txt; do
    [ -e "$filename" ] || continue
    # ... rest of the loop body
done

Reference: Bash Pitfalls

Answer 3

for file in Data/*.txt
do
    for ((i = 0; i < 3; i++))
    do
        name=${file##*/}
        base=${name%.txt}
        ./MyProgram.exe "$file" Logs/"${base}_Log$i.txt"
    done
done

The name=${file##*/} substitution (shell parameter expansion) removes the leading pathname up to the last /.

The base=${name%.txt} substitution removes the trailing .txt. It's a bit trickier if the extensions can vary.

Answer 4

You can use finds null separated output option with read to iterate over directory structures safely.

#!/bin/bash
find . -type f -print0 | while IFS= read -r -d $'\0' file; 
  do echo "$file" ;
done

So for your case

#!/bin/bash
find . -maxdepth 1 -type f  -print0 | while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done

additionally

#!/bin/bash
while IFS= read -r -d $'\0' file; do
  for ((i=0; i<=3; i++)); do
    ./MyProgram.exe "$file" 'Logs/'"`basename "$file"`""$i"'.txt'
  done
done < <(find . -maxdepth 1 -type f  -print0)

will run the while loop in the current scope of the script ( process ) and allow the output of find to be used in setting variables if needed

Answer 5

Run a command on each file

do something (echo) with all .txt files

for f in *.txt;  do echo ${f}; done;