Change elements in vector using multithreading in Rust

Question

I'm new in Rust and i'm trying to allocate computational work to threads.

I have vector of strings, i would want to create to each string one thread to do his job. There's simple code:

use std::thread;

fn child_job(s: &mut String) {
    *s = s.to_uppercase();
}

fn main() {
    // initialize
    let mut thread_handles = vec![];
    let mut strings = vec![
        "hello".to_string(),
        "world".to_string(),
        "testing".to_string(),
        "good enough".to_string(),
    ];

    // create threads
    for s in &mut strings {
        thread_handles.push(thread::spawn(|| child_job(s)));
    }

    // wait for threads
    for handle in thread_handles {
        handle.join().unwrap();
    }

    // print result
    for s in strings {
        println!("{}", s);
    }
}

I got errors while compile:

error[E0597]: `strings` does not live long enough
  --> src/main.rs:18:14
   |
18 |     for s in &mut strings {
   |              ^^^^^^^^^^^^
   |              |
   |              borrowed value does not live long enough
   |              argument requires that `strings` is borrowed for `'static`
...
31 | }
   | - `strings` dropped here while still borrowed

error[E0505]: cannot move out of `strings` because it is borrowed
  --> src/main.rs:28:14
   |
18 |     for s in &mut strings {
   |              ------------
   |              |
   |              borrow of `strings` occurs here
   |              argument requires that `strings` is borrowed for `'static`
...
28 |     for s in strings {
   |              ^^^^^^^ move out of `strings` occurs here

I can't understand what's wrong with lifetime of pointers and how i should fix that. For me it looks OK, because every thread get only one mutable pointer of string and doesn't affect the vector itself in any way.

Answer 1

Caesar's answer shows how to solve the problem using crossbeam's scoped threads. If you don't want to depend on crossbeam, then the approach of wrapping the values in Arc<Mutex<T>>, as shown in tedtanner's answer, is a reasonable general strategy.

However in this case the mutex is really unnecessary because the threads don't share the strings, either with each other or with the main thread. Locking is an artifact of using Arc, which is itself mandated by the static lifetime rather than a need for sharing. Although the locks are uncontended, they do add some overhead and are best avoided. In this case we can avoid both Arc and Mutex by moving each string to its respective thread, and retrieving the modified string once the threads finish.

This modification compiles and runs using just the standard library and safe code, and without requiring Arc or Mutex:

// ... child_job defined as in the question ...

fn main() {
    let strings = vec![
        "hello".to_string(),
        "world".to_string(),
        "testing".to_string(),
        "good enough".to_string(),
    ];

    // start the threads, giving them the strings
    let mut thread_handles = vec![];
    for mut s in strings {
        thread_handles.push(thread::spawn(move || {
            child_job(&mut s);
            s
        }));
    }

    // wait for threads and re-populate `strings`
    let strings = thread_handles.into_iter().map(|h| h.join().unwrap());

    // print result
    for s in strings {
        println!("{}", s);
    }
}

Playground

Answer 2

Rust doesn't know that your strings will last as long as your threads, so it won't pass a reference to them to the threads. Imagine if you passed a reference to a string to another thread, then the original thread thought it was done with that string and freed its memory. That would cause undefined behavior. Rust prevents this by requiring that the strings either be kept behind a reference-counted pointer (ensuring their memory doesn't get freed while they are still referenced somewhere) or that they have a 'static lifetime, meaning they are stored in the executable binary itself.

Also, Rust won't allow you to share a mutable reference across threads because it is unsafe (multiple threads could try to alter the referenced data at once). You want to use an std::sync::Arc in conjunction with a std::sync::Mutex. Your strings vector would become a Vec<Arc<Mutex<String>>>. Then, you could copy the Arc (using .clone()) and send that across threads. Arc is a pointer that keeps a reference count that gets incremented atomically (read: in a thread-safe way). The mutex allows a thread to lock the string temporarily so other threads can't touch it then unlock the string later (the thread can safely change the string while it is locked).

Your code would look something like this:

use std::thread;
use std::sync::{Arc, Mutex};

fn child_job(s: Arc<Mutex<String>>) {
    // Lock s so other threads can't touch it. It will get
    // unlocked when it goes out of scope of this function.
    let mut s = s.lock().unwrap();
    *s = s.to_uppercase();
}

fn main() {
    // initialize
    let mut thread_handles = Vec::new();
    let strings = vec![
        Arc::new(Mutex::new("hello".to_string())),
        Arc::new(Mutex::new("world".to_string())),
        Arc::new(Mutex::new("testing".to_string())),
        Arc::new(Mutex::new("good enough".to_string())),
    ];

    // create threads
    for i in 0..strings.len() {
        let s = strings[i].clone();
        thread_handles.push(thread::spawn(|| child_job(s)));
    }

    // wait for threads
    for handle in thread_handles {
        handle.join().unwrap();
    }

    // print result
    for s in strings {
        let s = s.lock().unwrap();
        println!("{}", *s);
    }
}

Answer 3

[Update] issue was resolved by completely uninstalling my http-server module:

which http-server - > directory

remove directory

and reinstalling http-server with brew rather than npm....

I later found this which is an exact replica of the issue I was seeing https://snyk.io/blog/open-source-npm-packages-colors-faker/, the npm install must be using the vulnerable method of [email protected] where as brews version doesn't use it.

http-server maintainers are aware of this and working to resolve it.