'Python Threading Worker Closure
I'm writing a script that contains multiple threads interconnected with queues. The lowest level thread is taking data from a UDP port. The other threads do various levels of processing on that data e.g.
Thread 3: Process application data
Thread 2: Process layer 4 packet headers
Thread 1: Listen for UDP packets
I get that normally you should use .task_done() to indicate that the work has been completed and that .join() will wait until that all tasks are done. However, I need to start these threads before the UDP data is being sent. So what should I do in the case that the queues will remain empty for some time before UDP data is received?
I think I have a couple of options:
- The UDP listener (Thread 1) has a timeout. at the end of the timeout I could put something that signifies the end in the queue, e.g. udp_queue.put(-1). Then the following threads can add further -1s to the other queues. Is there something better to use than a -1?
- Implement timeouts that increment on Queue.Empty() which are larger than the time out in the UDP listener (Thread 1).
I assume option 1 is the preference? My concern is that if I have some valid data in a queue that happens to equal -1 then I could end up closing the threads unintentionally?
Any thoughts or pointer appreciated.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
