Ring buffer log file on unix

Question

I'm trying to come up with a unix pipeline of commands that will allow me to log only the most recent n lines of a program's output to a text file.

The text file should never be more than n lines long. (it may be less when it is first filling the file)

It will be run on a device with limited memory/resources, so keeping the filesize small is a priority.

I've tried stuff like this (n=500):

program_spitting_out_text > output.txt
cat output.txt | tail -500 > recent_output.txt
rm output.txt

or

program_spitting_out_text | tee output.txt | tail -500 > recent_output.txt

Obviously neither works for my purposes...

Anyone have a good way to do this in a one-liner? Or will I have to write a script/utility?

Note: I don't want anything to do with dmesg and must use standard BSD unix commands. The "program_spitting_out_text" prints out about 60 lines/second, every second.

Thanks in advance!

Answer 1

Since apparently this basic feature (circular file) does not exist on GNU/Linux, and because I needed it to track logs on my Raspberry Pi with limited storage, I just wrote the code as suggest above!

Behold: circFS

Unlike other tools quoted on this post and other similar, the maximum size is arbitrary and only limited by the actual available storage. It does not rotate with several files, all is kept in the single file, which is rewritten on "release". You can have as many log files as needed in the virtual directory.

It is a single C file (~600 lines including comments), and it builds with a single compile line after having installed fuse development dependencies.

This first version is very basic (see the README), if you want to improve it with some of the TODOs (see the TODO) be welcome to submit pull requests.

As a joke, this is my first "write only" fuse driver! :-)