'Custom prefix for redis keys with Celery
I am using redis as a broker between Django and Celery. The redis instance I have access to is shared with many other applications and so the broker is not reliable (the redis keys it uses are deleted by others, the messages often get sent to workers in other applications). Changing redis database does not solve the problem (there are few databases and many applications).
How can I configure Celery to prefix all the keys it uses with a custom string? The docs mention ways to add prefixes to queue names, but that does not affect the redis keys. The underlying library (Kombu) does not seem to let the user prefix the keys it uses as far as I can tell.
Solution 1:[1]
The functionality to add prefix to all the redis keys has been added as part of this. Now you can configure it like this:
BROKER_URL = 'redis://localhost:6379/0'
celery = Celery('tasks', broker=BROKER_URL, backend=BROKER_URL)
celery.conf.broker_transport_options = {'global_keyprefix': "prefix"}
Solution 2:[2]
This is not supported by Celery yet. A pull request on this subject is currently stalled due to a lack of workforce:
Solution 3:[3]
You can just override the prefix value of your celery task.
@shared_task(bind=True)
def task(self, params):
self.backend.task_keyprefix = b'new-prefix'
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Ajay Gupta |
| Solution 2 | pintoch |
| Solution 3 | Grigori Kartashyan |