Celery chord failing with ValueError: not enough values to unpack (expected 2, got 1)

Question

I am running celery v5.2.3 in a docker container running ubuntu.

Here I am trying to get celery chords to work but every attempt I make give me:

    File "/usr/local/lib/python3.7/site-packages/celery/result.py", line 1056, in result_from_tuple
    res, nodes = r
ValueError: not enough values to unpack (expected 2, got 1)

After this error the chord keeps retrying with:

retry: Retry in 1.0s: ValueError('not enough values to unpack (expected 2, got 1)')

The tasks I am trying to run is as follows:

@celery_app.task(shared=False)
def add(x, y):
    return x + y


@celery_app.task(shared=False)
def tsum(numbers):
    return numbers

@celery_app.task(name="celery.test")
def test():
    x = chord([add.s(i, i) for i in range(10)], body=tsum.s())
    r = x.apply_async()
    r.get()

My sample runs for 9/10 iterations, but then fails.

The celery worker is running with:

celery -A scheduler worker -P eventlet -l info 

Can any of you tell me what I am doing wrong as I cannot find anything on the internet explaining this issue?

Answer 1

For any unlucky soul in the future that should experience this issue.

The issue was cause with us having a custom implementation of the mongo backend for celery. Here we defined the meta fields for the tasks as follows:

meta = {
            "_id": task_id,
            "status": status,
            "result": result,
            "date_done": datetime.utcnow(),
            "traceback": traceback,
            "children": self.encode(
                self.current_task_children(request),
            ),
            "task_name": task_name,
            "task_args": task_args,
            "task_kwargs": task_kwargs,
            "worker": worker,
        }

However as celery expects the children field to be a list of children or None, my issue was because the children field would always be populated by an empty list.

By changing children to:

"children": self.encode(
    self.current_task_children(request),
)
if len(self.current_task_children(request)) > 0
else None,

The issue would be solved.

I hope this will help if anyone should experience this