'Delegate task from Multiprocess to Sub Process in Python

Hi currently I am using python telepot to:

  1. Listen to incoming message
  2. Based on the message it will do a long operation to generate some image.

Problem: The message is being process 1 by 1.

So if 2 users send in the message at the same time, it will process and send the image 1 by 1.

Have started reading Multiprocessing, but it is not getting what I want. What I have achieve now.

  1. Switched to aiogram, listen to message
  2. Pass the message to Process(target=imageprocessing.incoming, args=(userMsg,))
  3. This kind of work, but every time a message comes in it will start a new process, it is kind of slow because it has to initialise some libraries etc before it process it.

Thus it is possible to:

  1. Start X amount of "sub-process" in the main script initially.
  2. Main script will listen for incoming message, queue it and pass it to one of the "sub-process"
  3. So that X amount of message simultaneously based on how many "sub-process" I have defined initially.

Have been self-learning python as a side hobby. Not sure if I have used the term correctly.

Try googling but can't find a solution for now, got a feeling I might be looking up the wrong keyword.

Anyone has any idea what should I look for?

Thank you very much.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source