'How to group multiple functions for parallel execution with Celery?
I wanted to group multiple functions for parallel execution in Django within the transaction.on_commit function -however, I got TypeError: cannot unpack non-iterable int object I have tried to resolve this, but I now think I'm misusing one of the args. This was my initial approach:
# tasks.py
def func_a(keys: tuple, link_set: dict):
# keys contain random integers as pk
return
def func_b(keys: tuple, link_set: dict):
return
def func_c(keys: tuple, link_set: dict):
return
I tried to call these in my signal like this:
@receiver(post_save, sender=Foo)
def start_task(sender, instance, created, **kwargs):
if created:
...
keys = (instance.pk, foo.pk)
link_set = {}
transaction.on_commit(lambda: group([
tasks.func_a.si(keys, link_set),
tasks.func_b.si(keys, link_set),
tasks.func_c.si(keys, link_set)
]).delay())
The 3 functions in tasks.py are independent of each other with no return value, putting them together into one incurs a little cost that I'd like to avoid- how best can I compose these functions for a faster response? Thanks
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
