'Getting information back from a process with a multiprocessing Queue

I am trying to play around with multiprocessing and I would like to communicate between Python's main thread and a subprocess with a Queue. Here is a quick test code I wrote that should get periodically some results generated by the subprocess:

from multiprocessing import Process, Queue
import time

def calculate(queue):  
    n = 0
    while n < 10:   
        n += 1
        queue.put(n)
        time.sleep(1)
    queue.put(0)

def queue_getter(queue):
    executing = True
    while executing:     
        while queue.qsize():
            n = queue.get()
            print(n)
            if n == 0:
                executing = False
        time.sleep(0.1)
    print('done')

queue = Queue()
p = Process(target=calculate, args=(queue,))
p.start()
queue_getter(queue)
p.join()
print('DONE')

This program just hangs forever, while replacing Process with threading.Thread gives the expected result:

1
2
3
4
5
6
7
8
9
10
0
done
DONE

How to make Process behave the same way as Thread in this situation?



Solution 1:[1]

Here's a simplified and more robust approach which is (almost) functionally identical to the OP's original except that is does not print the zero:

from multiprocessing import Manager
from concurrent.futures import ProcessPoolExecutor
import time

def calculate(q):
    for n in range(1, 11):
        q.put(n)
        time.sleep(1)
    q.put(0)

def queue_getter(q):
    while (n := q.get()):
        print(n)

def main():
    with Manager() as manager:
        q = manager.Queue()
        with ProcessPoolExecutor() as executor:
            executor.submit(calculate, q)
            queue_getter(q)

if __name__ == '__main__':
    main()

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1