skip to Main Content

I’ve followed the celery doc https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html to create an app and periodic tasks as below:

$ tree demo/
demo/
├── config.py
├── __init__.py
└── tasks.py

$ cat demo/__init__.py 
# -*- coding: utf-8 -*-
from celery import Celery

app = Celery('demo')
app.config_from_object('demo.config')

$ cat demo/config.py 
# -*- coding: utf-8 -*-
BROKER_URL = "redis://127.0.0.1:6379"
CELERY_TIMEZONE='UTC'
CELERY_IMPORTS = [
    "demo.tasks",
]

$ cat demo/tasks.py 
from demo import app
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
    sender.add_periodic_task(3.0, say.s(), name='say hello every 3s')

@app.task
def say():
    print("Hello!")

And then run celery beat as below:

$ celery beat -A demo -l info --max-interval 10
celery beat v4.3.0 (rhubarb) is starting.
__    -    ... __   -        _
LocalTime -> 2019-12-12 16:26:41
Configuration ->
    . broker -> redis://127.0.0.1:6379//
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%INFO
    . maxinterval -> 10.00 seconds (10.0s)
[2019-12-12 16:26:41,234: INFO/MainProcess] beat: Starting...

Wait for a while, there isn’t any task scheduled.

However if I change to set CELERYBEAT_SCHEDULE in config, it can work well. I do that by changing demo/config.py and demo/tasks.py as below:

$ cat demo/config.py 
# -*- coding: utf-8 -*-

from datetime import timedelta

BROKER_URL = "redis://127.0.0.1:6379"

CELERY_TIMEZONE='UTC'
CELERY_IMPORTS = [
    "demo.tasks",
]

CELERYBEAT_SCHEDULE = {
    'say hello every 10 seconds': {
         'task': 'demo.tasks.say',
         'schedule': timedelta(seconds=3),
    },
}
$ cat demo/tasks.py 
from demo import app

@app.task
def say():
    print("Hello!")

Then run celery beat with the same command as before, the periodic tasks can be scheduled every 3 seconds as expected.

What’s wrong with my previous setup?

3

Answers


  1. Chosen as BEST ANSWER

    FYI, I figured out another solution, changing the decorator @app.on_after_configure.connect to @app.on_after_finalize.connect can make it work. Though I don't know the exact reason at this moment.


  2. You must specify the scheduler by --scheduler key according this userguide. For example you could try something like this code below:

    celery beat -A demo -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
    

    P.S. I don’t sure that --max-interval not interfere with your schedule period. Think is better to remove this option. IMHO.

    Login or Signup to reply.
  3. Celery 4.x does not use the upper-case config variables – see the New lowercase settings section of the Celery documentation. Scheduler-specific configuration arguments are listed here. So, try to modify your code to have beat_schedule = { instead of CELERYBEAT_SCHEDULE = {. The link you say you followed also uses lower-case names…

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search