'How to indicate progress of huge rsync, restic processes in a Gitlab job?

I'm using rsync and restic to backup lots of directories and files from some servers and these processes are defined in Gitlab jobs.

I've tried using different flags to make their output smaller but still indicate some progress... But the data is so big that it still reached Gitlab's "log limit".

So now it logs just the start and the end of the hour-long process and there is no way to know whether it is running okay or not without looking at the CPU.

But I'm a rookie so maybe I've just missed some useful flags.

Could you help me find a way to indicate progress in a tidy manner?

For starters it would be nice to just log after every x thousandth files that "x thousand files have been processsed so far".

Or log every 100th processed directory's path. This is even better because if it's being processed in alphabetical order it would be able to tell me where is the process currently.

Thanks in advance!



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source