What I have: I run celery with rabbitMQ as a broker and redis as the result backend. I have an app that sends tasks and workers that process the tasks. I deployed this as follow: The app, redis, rabbitMQ and…
I have django 3.2.7, celery 5.2.1, redis 3.5.3 I have next celery settings. (REDIS_PASSWORD) is env variable: CELERY_BROKER_URL = f'redis://:{REDIS_PASSWORD}@redis:6379/4' CELERY_BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 3600} CELERY_RESULT_BACKEND = f'redis://:{REDIS_PASSWORD}@redis:6379/1' CELERY_ACCEPT_CONTENT = ['application/json'] But when I start my docker-compose app, it shows me…
I have "locker" decorator: def lock_task(func): def wrapper(*args, **kwargs): if redis.set(func.__name__, 'lock', nx=True): try: result = func(*args, **kwargs) finally: redis.delete(func.__name__) return result or True else: return 'Skipped' return wrapper Also I have celery-task with my decorator: @celery_app.task @lock_task def test():…
After playing with some "defect" scenarios with celery (Redis being a broker for whatever it worth) we came to understanding that there is effectively no sense in setting acks_late=true without simultaneous setting of task_reject_on_worker_lost=true because the task won't be rescheduled…
I'm trying to run my tasks and I'm currently using Celery for this alongside Django and DjangoRestFramework This is how I've set things up, root directory is also called backend . ├── backend │ ├── asgi.py │ ├── celery.py │…
I have a series of celery workers which carry out tasks, stored using REDIS; in particular I have three categories of workers which respectively carry out three categories of tasks. The tasks belonging to each of these categories require access…
I’m setting up a distributed Airflow cluster where everything else except the celery workers are run on one host and processing is done on several hosts. The airflow2.0 setup is configured using the yaml file given at the Airflow documentation…
Using celery with redis in my django app Everything was working fine until I ran into a problem. The location of the redis files was changed and redis could not access them. After searching, it turns out that this is…
I have taken a free trial for Redis and it gave me an endpoint with a password. I haven't done anything with Redis or celery before so I really don't have any idea how it works. From the Docs of…
I'm using django with celery, celery beat, and redis to run periodic tasks. The problem I'm having is that tasks registered in celery are received but not executed. Also, the task stops in the middle even though I have not…