'How to copy a folder recursively in Rust?
From what I see in the documentation, there is no out-of-the-box solution.
Solution 1:[1]
You can use the fs_extra crate that I have written. This crate expands on the standard library std::fs and std::io modules.
It can (among other things):
- Copy files (optionally with information about the progress).
- Copy directories recursively (optionally with information about the progress).
Solution 2:[2]
use std::fs;
use std::path::{Path, PathBuf};
pub fn copy<U: AsRef<Path>, V: AsRef<Path>>(from: U, to: V) -> Result<(), std::io::Error> {
let mut stack = Vec::new();
stack.push(PathBuf::from(from.as_ref()));
let output_root = PathBuf::from(to.as_ref());
let input_root = PathBuf::from(from.as_ref()).components().count();
while let Some(working_path) = stack.pop() {
println!("process: {:?}", &working_path);
// Generate a relative path
let src: PathBuf = working_path.components().skip(input_root).collect();
// Create a destination if missing
let dest = if src.components().count() == 0 {
output_root.clone()
} else {
output_root.join(&src)
};
if fs::metadata(&dest).is_err() {
println!(" mkdir: {:?}", dest);
fs::create_dir_all(&dest)?;
}
for entry in fs::read_dir(working_path)? {
let entry = entry?;
let path = entry.path();
if path.is_dir() {
stack.push(path);
} else {
match path.file_name() {
Some(filename) => {
let dest_path = dest.join(filename);
println!(" copy: {:?} -> {:?}", &path, &dest_path);
fs::copy(&path, &dest_path)?;
}
None => {
println!("failed: {:?}", path);
}
}
}
}
}
Ok(())
}
Other answers don't actually show how to do it, they just say how you might do it; here is a concrete example.
As discussed in the other answers, the relevant APIs are fs::create_dir_all, fs::copy and fs::metadata.
There is no 'all in one' standard library API for this.
Solution 3:[3]
Simplest code that I've found works:
use std::{io, fs};
fn copy_dir_all(src: impl AsRef<Path>, dst: impl AsRef<Path>) -> io::Result<()> {
fs::create_dir_all(&dst)?;
for entry in fs::read_dir(src)? {
let entry = entry?;
let ty = entry.file_type()?;
if ty.is_dir() {
copy_dir_all(entry.path(), dst.as_ref().join(entry.file_name()))?;
} else {
fs::copy(entry.path(), dst.as_ref().join(entry.file_name()))?;
}
}
Ok(())
}
Solution 4:[4]
You could recursively walk the directory structure using std::fs::walk_dir, which yields an iterator in a Result, and for each Path, check if it's a file using the is_file() method provided by the PathExtensions extension trait. If it is, then use std::io::fs::copy to actually copy the file.
Solution 5:[5]
fs_extra simply didn't work for me. Also it has confusing options, and the overall quality is suspicious (e.g. they think 64000 bytes is 64 kB).
Anyway an alternative which worked is copy_dir which contains a single function to do this, with no options. It won't overwrite existing directories, but you can probably modify the code to allow that pretty easily.
Solution 6:[6]
A complete example of a program copying a directory content, using the crate walkdir:
Let's start by listing the current files.
(Note: on windows tree /F list all files and directories)
$ tree /F
? Cargo.lock
? Cargo.toml
?
????in
? ? b.txt
? ?
? ????dir_a
? a.txt
?
????src
main.rs
Rust sources
# File Cargo.toml
[package]
name = "plop"
version = "0.1.0"
edition = "2021"
[dependencies]
walkdir = "2" # latest version when writing this answer is 2.3
// File src/main.rs
use std::error::Error;
use std::fs;
use std::io;
use std::path::PathBuf;
use walkdir::WalkDir;
fn main() -> Result<(), Box<dyn Error>> {
let in_dir = PathBuf::from("in");
let out_dir = PathBuf::from("out");
for entry in WalkDir::new(&in_dir) {
let entry = entry?;
let from = entry.path();
let to = out_dir.join(from.strip_prefix(&in_dir)?);
println!("\tcopy {} => {}", from.display(), to.display());
// create directories
if entry.file_type().is_dir() {
if let Err(e) = fs::create_dir(to) {
match e.kind() {
io::ErrorKind::AlreadyExists => {}
_ => return Err(e.into()),
}
}
}
// copy files
else if entry.file_type().is_file() {
fs::copy(from, to)?;
}
// ignore the rest
else {
eprintln!("copy: ignored symlink {}", from.display());
}
}
Ok(())
}
Let's build and run it
$ cargo run
Compiling winapi v0.3.9
Compiling winapi-util v0.1.5
Compiling same-file v1.0.6
Compiling walkdir v2.3.2
Compiling plop v0.1.0 (C:\Users\Sam\Development\plop)
Finished dev [unoptimized + debuginfo] target(s) in 6.33s
Running `target\debug\plop.exe`
copy in => out\
copy in\b.txt => out\b.txt
copy in\dir_a => out\dir_a
copy in\dir_a\a.txt => out\dir_a\a.txt
Everything went well, let's again check files and directories.
$ tree /F
? Cargo.lock
? Cargo.toml
?
????in
? ? b.txt
? ?
? ????dir_a
? a.txt
?
????out
? ? b.txt
? ?
? ????dir_a
? a.txt
?
????src
? main.rs
|
... # more content we can ignore, like the 'target' build directory
Solution 7:[7]
I borrowed Doug's solution, but I think rayon makes the best bed sheets, so I did it with rayon. This is quick and dirty in that it doesn't handle any error cases (threads just crash instead of finishing) but you'll know when that happens if you use this as-is as it won't hide it. Should be easy to add your own error handling if you want it. I'm also a very amateurish programmer and my code shouldn't be used by anybody with any sense. I made it in such a way that you could use the indicatif crate to very easily add a nice looking progress bar.
When doing network copies of large folders of a ton of small files, this does the job pretty quickly in my tests.
No platform specific bits are used, so it should run anywhere that rust std compiles.
use clap::Parser;
use rayon::Scope;
use std::fs::{self, DirEntry};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use crossbeam::queue::SegQueue;
use rayon::iter::{ParallelIterator, IntoParallelRefIterator};
#[derive(Debug, Parser)]
#[clap(name = "mtcopy", about = "Multi-threaded copy...in rust!")]
struct Opt {
/// Source directory
source: String,
/// Destination directory
dest: String,
}
fn main() {
let args = Opt::parse();
copy(&args.source, &args.dest);
}
fn copy<U: AsRef<Path>, V: AsRef<Path>>(from: &U, to: &V) {
let start = std::time::Instant::now();
let source = PathBuf::from(from.as_ref());
let dest = PathBuf::from(to.as_ref());
let total_size = AtomicU64::new(0);
// Could possibly use dashmap for the bag instead, slower but it can be used to avoid duplicate files from e.g. symlinks
let bag = SegQueue::new();
rayon::scope(|s| scan(&source, &bag, s, &total_size));
// Convert to a vec so that we can rayon it and throw the bag away
let files = bag.into_iter().collect::<Vec<_>>();
let total_size = total_size.into_inner();
files.par_iter().for_each(|entry| {
let entry = entry.as_ref().unwrap();
let spath = entry.path();
let stem = spath.strip_prefix(&source).unwrap();
let dpath = dest.join(&stem);
let size = entry.metadata().unwrap().len();
fs::create_dir_all(&dpath.parent().unwrap()).unwrap();
// println!(" copy: {:?} -> {:?}", &path, &dpath);
fs::copy(&spath, &dpath).unwrap();
});
println!("Copied {} files, {} bytes in {}", files.len(), total_size, start.elapsed().as_secs());
}
fn scan<'a, U: AsRef<Path>>(src: &U, bag: &'a SegQueue<Result<DirEntry, std::io::Error>>, s: &Scope<'a>, total_size: &'a AtomicU64) {
let dir = fs::read_dir(src).unwrap();
dir.into_iter().for_each(|entry| {
let info = entry.as_ref().unwrap();
let path = info.path();
if path.is_dir() {
s.spawn(move |s| scan(&path, bag, s, total_size))
} else {
// println!("{}", path.as_os_str().to_string_lossy());
let filelength = info.metadata().unwrap().len();
total_size.fetch_add(filelength, Ordering::SeqCst);
bag.push(entry)
}
})
}
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Shepmaster |
| Solution 2 | Doug |
| Solution 3 | Simon Buchan |
| Solution 4 | reducing activity |
| Solution 5 | Timmmm |
| Solution 6 | |
| Solution 7 |