'Parallel copy to move large amount of Data in linux
I have folder which needs to be migrated to NFS location and its around 20T. when I try to rsync(or using parallel rsync) its taking lot of time. even with parallel rsync. is there any way we can improve copy rate? I just want to move entire directory even no need of syncing:
Tried the following option but with this its taking very long:
rsync -avzm --stats --safe-links --ignore-existing --dry-run --human-readable /<dir>/ /<nfs path> > /tmp/sync.log
/tmp/sync.log | parallel --will-cite -j 5 rsync -avzm --relative --stats --safe-links --ignore-existing --human-readable {} /<nfs path (dest dir)> > /tmp/result.log
I think its overwriting and taking lot than regular.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|