How to create multiprocess with regression function?

Question

I'm trying to build a regression function that call itself in a new process. The new process should not stop the parent process nor wait for it to finish, that is why I don't use join(). Do you have another way to create regression function with multi-process. I use the following code:

import multiprocessing as mp
import concurrent.futures
import time
def do_something(c, seconds, r_list):
  c += 1 # c is a counter that all processes should use
       # such that no more than 20 processes are created. 
  print(f"Sleeping {seconds} second(s)...")

  if c < 20:
    P_V = mp.Value('d', 0.0, lock=False)
    p = mp.Process(group=None, target=do_something, args=(c, 1, r_list,))
    p.start()
    if not p.is_alive():
        r_list.append(P_V.value)

  time.sleep(seconds)
  print(f"Done Sleeping...{seconds}")
  return f"Done Sleeping...{seconds}"

if __name__ == '__main__':
  C = 0 # C is a counter that all processes should use
      # such that no more than 20 processes are created. 
  Result_list = [] # results that come from all processes are saved here
  Result_list.append(do_something(C, 1, Result_list))

Notice that results from all processes should be compared at the end. In fact, this code is working well but the child processes, which are created in the recursive method, do not print anything, the list "Result_list" contains only one item from the first call, and C=0 at the end, any idea why?

Answer 1

Here's a simplified example of what I think you're trying to do (side note: launching processes recursively is a great way to accidentally create a "fork bomb". It is extremely more common to create multiple processes in some sort of loop instead)

from multiprocessing import Process, Queue
from time import sleep
from os import getpid

def foo(n_procs, return_Q, arg):
    if __name__ == "__main__": #don't actually run the body of foo in the "main" process, just start the recursion
        Process(target=foo, args=(n_procs, return_Q, arg)).start()
    else:
        n_procs -= 1
        if n_procs > 0:
            Process(target=foo, args=(n_procs, return_Q, arg)).start()
        sleep(arg)
        print(f"{getpid()} done sleeping {arg} seconds")
        return_Q.put(f"{getpid()} done sleeping {arg} seconds") #put the result to a queue so we can get it in the main process

if __name__ == "__main__":
    q = Queue()
    foo(10, q, 2)

    sleep(10) #do something else in the meantime

    results = []
    #while not q.empty(): #usually better to just know how many results you're expecting as q.empty can be unreliable
    for _ in range(10):
        results.append(q.get())

    print("mp results:")
    print("\n".join(results))