'Is it possible appending remote files into local tar file using python's tarfile module?

I'm trying to create local tar file out of remote files,

I have managed doing that in command line with following two commands:

run the following command for each full_path_to_input_file that I want to add:

ssh <remote_computer_name> 'tar uvf <full_path_to_tar.tar> <full_path_to_input_file>'

and after that run (to compress the file):

gzip <full_path_to_input_file>

So, I think I can solve the problem using os.system("...") python command with the above two commands, But I want to know if maybe python's tarfile module supports that? to avoid executing external commands in a shell.

I have seen that tarfile module has the add method: *TarFile.add(name, arcname=None, recursive=True, , filter=None) but it seems that name can't be remote, am I right? if not, I would really appreciate some code snippet showing how to use that method on remote files.

The background is this: I have a system, build of multiple computers (the exact number varies from system to system), each of which is generating all kinds of log files. On some erroneous states I need to collect all logs from all computers of the system (suppose: master, slave-1, slave-2, etc.). Until now, the code used to rsync the remote files to some local dir, and after collecting all the log files tarfile.add was used on this directory to create tar file. This solution is problematic since there could be a situation that HD of master (the local computer) is nearly full, no place to both copy all the logs on it and create tar file, So I need the create the tar file "on the fly" - appending each remote file to the local tar.

So, if TarFile.add can't support this requirements, maybe you can suggest me some other solution? By the way, I'm using Python 2.7 and tarfile version is 0.9.0.

Thank you so much in advance! :)



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source