How do you run multiprocessing pool.starmap outside of __main__ function?

Question

All the documentation I have read mentioned that the Pool should be surrounded by a check that it's in main. Otherwise, there is potential for an infinite loop.

What I see Online to Do:

if __name__ == "__main__":
     with Pool(processes=5) as pool:
          output = pool.starmap(test_func, list(tuples))

However, I am running the multiprocessing library in a sub-module (not 'main') and not having any errors (note running it thru a Juypter-notebook). I do have an infinite loop when trying to run some integration tests I made with the unittest library.

How can I run the multiprocessing step in a sub-module?

How I am Currently Doing It:

with Pool(processes=5) as pool:
     output = pool.starmap(test_func, list(tuples))

Answer 1

Since I run most of my code in a Jupyter notebook I have run into this same problem. From what I have read Jupyter has a problem with namespace/ memory when trying to run a function that is in the same notebook inside jupyter.

The fix for me was to place the function I wanted to run into a module.py file and import the function and run the pool that way. IDKY but that works when running in Jupyter.