'Shell one liner to prepend to a file
This is probably a complex solution.
I am looking for a simple operator like ">>", but for prepending.
I am afraid it does not exist. I'll have to do something like
mv myfile tmp cat myheader tmp > myfile
Anything smarter?
Solution 1:[1]
This still uses a temp file, but at least it is on one line:
echo "text" | cat - yourfile > /tmp/out && mv /tmp/out yourfile
Solution 2:[2]
echo '0a
your text here
.
w' | ed some_file
ed is the Standard Editor! http://www.gnu.org/fun/jokes/ed.msg.html
Solution 3:[3]
John Mee: your method is not guaranteed to work, and will probably fail if you prepend more than 4096 byte of stuff (at least that's what happens with gnu awk, but I suppose other implementations will have similar constraints). Not only will it fail in that case, but it will enter an endless loop where it will read its own output, thereby making the file grow until all the available space is filled.
Try it for yourself:
exec 3<>myfile && awk 'BEGIN{for(i=1;i<=1100;i++)print i}{print}' myfile >&3
(warning: kill it after a while or it will fill the filesystem)
Moreover, it's very dangerous to edit files that way, and it's very bad advice, as if something happens while the file is being edited (crash, disk full) you're almost guaranteed to be left with the file in an inconsistent state.
Solution 4:[4]
Not possible without a temp file, but here goes a oneliner
{ echo foo; cat oldfile; } > newfile && mv newfile oldfile
You can use other tools such as ed or perl to do it without temp files.
Solution 5:[5]
If you need this on computers you control, install the package "moreutils" and use "sponge". Then you can do:
cat header myfile | sponge myfile
Solution 6:[6]
It may be worth noting that it often is a good idea to safely generate the temporary file using a utility like mktemp, at least if the script will ever be executed with root privileges. You could for example do the following (again in bash):
(tmpfile=`mktemp` && { echo "prepended text" | cat - yourfile > $tmpfile && mv $tmpfile yourfile; } )
Solution 7:[7]
Using a bash heredoc you can avoid the need for a tmp file:
cat <<-EOF > myfile
$(echo this is prepended)
$(cat myfile)
EOF
This works because $(cat myfile) is evaluated when the bash script is evaluated, before the cat with redirect is executed.
Solution 8:[8]
assuming that the file you want to edit is my.txt
$cat my.txt
this is the regular file
And the file you want to prepend is header
$ cat header
this is the header
Be sure to have a final blank line in the header file.
Now you can prepend it with
$cat header <(cat my.txt) > my.txt
You end up with
$ cat my.txt
this is the header
this is the regular file
As far as I know this only works in 'bash'.
Solution 9:[9]
When you start trying to do things that become difficult in shell-script, I would strongly suggest looking into rewriting the script in a "proper" scripting language (Python/Perl/Ruby/etc)
As for prepending a line to a file, it's not possible to do this via piping, as when you do anything like cat blah.txt | grep something > blah.txt, it inadvertently blanks the file. There is a small utility command called sponge you can install (you do cat blah.txt | grep something | sponge blah.txt and it buffers the contents of the file, then writes it to the file). It is similar to a temp file but you dont have to do that explicitly. but I would say that's a "worse" requirement than, say, Perl.
There may be a way to do it via awk, or similar, but if you have to use shell-script, I think a temp file is by far the easiest(/only?) way..
Solution 10:[10]
Like Daniel Velkov suggests, use tee.
To me, that's simple smart solution:
{ echo foo; cat bar; } | tee bar > /dev/null
Solution 11:[11]
EDIT: This is broken. See Weird behavior when prepending to a file with cat and tee
The workaround to the overwrite problem is using tee:
cat header main | tee main > /dev/null
Solution 12:[12]
sed -i -e "1s/^/new first line\n/" old_file.txt
Solution 13:[13]
The one which I use. This one allows you to specify order, extra chars, etc in the way you like it:
echo -e "TEXTFIRSt\n$(< header)\n$(< my.txt)" > my.txt
P.S: only it's not working if files contains text with backslash, cause it gets interpreted as escape characters
Solution 14:[14]
Mostly for fun/shell golf, but
ex -c '0r myheader|x' myfile
will do the trick, and there are no pipelines or redirections. Of course, vi/ex isn't really for noninteractive use, so vi will flash up briefly.
Solution 15:[15]
With $( command ) you can write the output of a command into a variable. So I did it in three commands in one line and no temp file.
originalContent=$(cat targetfile) && echo "text to prepend" > targetfile && echo "$originalContent" >> targetfile
Solution 16:[16]
Why not simply use the ed command (as already suggested by fluffle here)?
ed reads the whole file into memory and automatically performs an in-place file edit!
So, if your file is not that huge ...
# cf. "Editing files with the ed text editor from scripts.",
# http://wiki.bash-hackers.org/doku.php?id=howto:edit-ed
prepend() {
printf '%s\n' H 1i "${1}" . wq | ed -s "${2}"
}
echo 'Hello, world!' > myfile
prepend 'line to prepend' myfile
Yet another workaround would be using open file handles as suggested by Jürgen Hötzel in Redirect output from sed 's/c/d/' myFile to myFile
echo cat > manipulate.txt
exec 3<manipulate.txt
# Prevent open file from being truncated:
rm manipulate.txt
sed 's/cat/dog/' <&3 > manipulate.txt
All this could be put on a single line, of course.
Solution 17:[17]
A variant on cb0's solution for "no temp file" to prepend fixed text:
echo "text to prepend" | cat - file_to_be_modified | ( cat > file_to_be_modified )
Again this relies on sub-shell execution - the (..) - to avoid the cat refusing to have the same file for input and output.
Note: Liked this solution. However, in my Mac the original file is lost (thought it shouldn't but it does). That could be fixed by writing your solution as: echo "text to prepend" | cat - file_to_be_modified | cat > tmp_file; mv tmp_file file_to_be_modified
Solution 18:[18]
Here's what I discovered:
echo -e "header \n$(cat file)" >file
Solution 19:[19]
sed -i -e '1rmyheader' -e '1{h;d}' -e '2{x;G}' myfile
Solution 20:[20]
WARNING: this needs a bit more work to meet the OP's needs.
There should be a way to make the sed approach by @shixilun work despite his misgivings. There must be a bash command to escape whitespace when reading a file into a sed substitute string (e.g. replace newline characters with '\n'. Shell commands vis and cat can deal with nonprintable characters, but not whitespace, so this won't solve the OP's problem:
sed -i -e "1s/^/$(cat file_with_header.txt)/" file_to_be_prepended.txt
fails due to the raw newlines in the substitute script, which need to be prepended with a line continuation character () and perhaps followed by an &, to keep the shell and sed happy, like this SO answer
sed has a size limit of 40K for non-global search-replace commands (no trailing /g after the pattern) so would likely avoid the scary buffer overrun problems of awk that anonymous warned of.
Solution 21:[21]
You can use perl command line:
perl -i -0777 -pe 's/^/my_header/' tmp
Where -i will create an inline replacement of the file and -0777 will slurp the whole file and make ^ match only the beginning. -pe will print all the lines
Or if my_header is a file:
perl -i -0777 -pe 's/^/`cat my_header`/e' tmp
Where the /e will allow an eval of code in the substitution.
Solution 22:[22]
If you're scripting in BASH, actually, you can just issue:
cat - yourfile /tmp/out && mv /tmp/out yourfile
That's actually in the Complex Example you yourself posted in your own question.
Solution 23:[23]
If you have a large file (few hundred kilobytes in my case) and access to python, this is much quicker than cat pipe solutions:
python -c 'f = "filename"; t = open(f).read(); open(f, "w").write("text to prepend " + t)'
Solution 24:[24]
A solution with printf:
new_line='the line you want to add'
target_file='/file you/want to/write to'
printf "%s\n$(cat ${target_file})" "${new_line}" > "${target_file}"
You could also do:
printf "${new_line}\n$(cat ${target_file})" > "${target_file}"
But in that case you have to be sure there aren’t any % anywhere, including the contents of target file, as that can be interpreted and screw up your results.
Solution 25:[25]
current=`cat my_file` && echo 'my_string' > my_file && echo $current >> my_file
where "my_file" is the file to prepend "my_string" to.
Solution 26:[26]
I'm liking @fluffle's ed approach the best. After all, any tool's command line switches vs scripted editor commands are essentially the same thing here; not seeing a scripted editor solution "cleanliness" being any lesser or whatnot.
Here's my one-liner appended to .git/hooks/prepare-commit-msg to prepend an in-repo .gitmessage file to commit messages:
echo -e "1r $PWD/.gitmessage\n.\nw" | ed -s "$1"
Example .gitmessage:
# Commit message formatting samples:
# runlevels: boot +consolekit -zfs-fuse
#
I'm doing 1r instead of 0r, because that will leave the empty ready-to-write line on top of the file from the original template. Don't put an empty line on top of your .gitmessage then, you will end up with two empty lines. -s suppresses diagnostic info output of ed.
In connection with going through this, I discovered that for vim-buffs it is also good to have:
[core]
editor = vim -c ':normal gg'
Solution 27:[27]
variables, ftw?
NEWFILE=$(echo deb http://mirror.csesoc.unsw.edu.au/ubuntu/ $(lsb_release -cs) main universe restricted multiverse && cat /etc/apt/sources.list)
echo "$NEWFILE" | sudo tee /etc/apt/sources.list
Solution 28:[28]
I think this is the cleanest variation of ed:
cat myheader | { echo '0a'; cat ; echo -e ".\nw";} | ed myfile
as a function:
function prepend() { { echo '0a'; cat ; echo -e ".\nw";} | ed $1; }
cat myheader | prepend myfile
Solution 29:[29]
IMHO there is no shell solution (and will never be one) that would work consistently and reliably whatever the sizes of the two files myheader and myfile. The reason is that if you want to do that without recurring to a temporary file (and without letting the shell recur silently to a temporary file, e.g. through constructs like exec 3<>myfile, piping to tee, etc.
The "real" solution you are looking for needs to fiddle with the filesystem, and so it's not available in userspace and would be platform-dependent: you're asking to modify the filesystem pointer in use by myfile to the current value of the filesystem pointer for myheader and replace in the filesystem the EOF of myheader with a chained link to the current filesystem address pointed by myfile. This is not trivial and obviously can not be done by a non-superuser, and probably not by the superuser either... Play with inodes, etc.
You can more or less fake this using loop devices, though. See for instance this SO thread.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow