How to define Celery Crontab Schedules in an external JSON/YAML file?

Question

Celery allows defining scheduled tasks by adding entries to the app.conf.beat_schedule. An example (based on link)

app.conf.beat_schedule = {
    # Executes every Monday morning at 7:30 a.m.
    'do-task-every-monday-morning': {
        'task': 'tasks.monday_morning_task',
        'schedule': crontab(hour=7, minute=30, day_of_week=1),
        'args': (18, 24),
    },
}

Is there a way to define the scheduled tasks in an external JSON file and load the file into Celery? For example,

{
    "do-task-every-monday-morning": {
        "task": "tasks.monday_morning_task",
        "schedule": "30 7 * * 1",
        "args": [18, 24]
    },
    "do-task-every-friday-afternoon": {
        "task": "tasks.friday_afternoon_task",
        "schedule": "30 14 * * 5"
    }
}

Any help would be appreciated.

Thanks in advance.

Answer 1

The part crontab(hour=7, minute=30, day_of_week=1), is a function call so you can't just write that in JSON and parse json to make it work in Python code.

You would need to add an intermediate step where you parse your schedule syntax "30 7 * * 1" into a value by calling the crontab function, and then set that value on the schedule key.

You would have a similar problem for the 'args': (18, 24), part, since the python tuple (18, 24) has no corresponding JSON so even if you wrote [18, 24] in your JSON file, you would have to interpret that with your own code and convert into the tuple.

Someone may already have done this so it could be worth a search on Github or elsewhere for code, but it's not a big job to implement.