'Why does program execution freeze when using ThreadPoolExecutor?
I have a script that transfers data throw the web constantly via API. My code works perfectly fine for about two hours or more, but then for no reason, the program freezes. API on both sides is still responding, but my collector stops doing anything. I have no errors, and the logs look ok.
Here is my code (i omitted some parts):
if __name__ == '__main__':
while True:
try:
source_data_list = src.get_source_data_list()
with concurrent.futures.ThreadPoolExecutor(max_workers=os.environ['THREAD_COUNT_GET']) as get_pool:
threads_in_get_pool = [get_pool.submit(src.get_data, source_data) for source_data in source_data_list]
with concurrent.futures.ThreadPoolExecutor(max_workers=os.environ['THREAD_COUNT_SEND']) as send_pool:
threads_in_send_pool = list()
for future in concurrent.futures.as_completed(threads_in_get_pool):
threads_in_send_pool.append(send_pool.submit(src.send_data, future.result())
except Exception as er:
cprint(er, 'red')
I understand that I give a little information, but I wonder about general reasons.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
