Question posted in
The official documentation can be found here.

Redis – task receive but doesn't excute

I’m learning Django. recently I tried to use celery. the problem that I’m facing with it is tasks are receiving but they don’t execute.

settings.py:

# Celery application definition
CELERY_BROKER_URL = 'redis://localhost:6379'
CELERY_RESULT_BACKEND = 'redis://localhost:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'Asia/Tehran'

celery.py:

from __future__ import absolute_import, unicode_literals

import os
from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myshop.settings')
app = Celery('myshop')
app.config_from_object('django.conf.settings', namespace='CELERY')
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

tasks.py:

from celery import task, shared_task
from django.core.mail import send_mail
from .models import Order

import logging

# Get an instance of a logger
logger = logging.getLogger(__name__)


@shared_task()
def order_created(order_id):
    logger.error('msg received')
    """
    Task to send an e-mail notification when an order is
    successfully created.
    """

    order = Order.objects.get(id=order_id)

    order = Order.objects.get(id=order_id)
    subject = f'Order nr. {order.id}'
    message = f'Dear {order.first_name},nn' 
              f'You have successfully placed an order.' 
              f'Your order ID is {order.id}.'
    mail_sent = send_mail(subject,
                          message,
                          '[email protected]',
                          [order.email])
    logger.error('mail sent')
    return mail_sent

running task in views.py:

# launch asynchronous task
            logger.error('before task')
            order_created.delay(order.id)
            logger.error('after task')

cmd log after running celery with celery -A myshop worker -l info:

(venv) E:Workspacedjango-shopmyshop>celery -A myshop worker -l info

 -------------- celery@DESKTOP-F7E0RGJ v4.4.2 (cliffs)
--- ***** -----
-- ******* ---- Windows-10-10.0.18362-SP0 2020-06-23 03:37:15
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app:         myshop:0x4b93df0
- ** ---------- .> transport:   redis://localhost:6379//
- ** ---------- .> results:     redis://localhost:6379/
- *** --- * --- .> concurrency: 8 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
 -------------- [queues]
                .> celery           exchange=celery(direct) key=celery


[tasks]
  . myshop.celery.debug_task
  . orders.tasks.order_created

[2020-06-23 03:37:15,800: INFO/MainProcess] Connected to redis://localhost:6379//
[2020-06-23 03:37:15,823: INFO/MainProcess] mingle: searching for neighbors
[2020-06-23 03:37:16,320: INFO/SpawnPoolWorker-2] child process 15856 calling self.run()
[2020-06-23 03:37:16,328: INFO/SpawnPoolWorker-1] child process 13464 calling self.run()
[2020-06-23 03:37:16,332: INFO/SpawnPoolWorker-4] child process 13528 calling self.run()
[2020-06-23 03:37:16,337: INFO/SpawnPoolWorker-3] child process 4660 calling self.run()
[2020-06-23 03:37:16,344: INFO/SpawnPoolWorker-8] child process 9040 calling self.run()
[2020-06-23 03:37:16,346: INFO/SpawnPoolWorker-5] child process 17044 calling self.run()
[2020-06-23 03:37:16,358: INFO/SpawnPoolWorker-7] child process 16480 calling self.run()
[2020-06-23 03:37:16,366: INFO/SpawnPoolWorker-6] child process 16876 calling self.run()
[2020-06-23 03:37:16,885: INFO/MainProcess] mingle: all alone
[2020-06-23 03:37:16,913: WARNING/MainProcess] e:workspacedjango-shopvenvlibsite-packagesceleryfixupsdjango.py:202: UserWarning: Using settings.DEBUG leads to a memory
            leak, never use this setting in production environments!
  warnings.warn('''Using settings.DEBUG leads to a memory
[2020-06-23 03:37:16,914: INFO/MainProcess] celery@DESKTOP-F7E0RGJ ready.
[2020-06-23 03:37:16,918: INFO/MainProcess] Received task: orders.tasks.order_created[1d366fb3-262b-4d28-92ab-8a7b85f1deff]
[2020-06-23 03:37:16,921: INFO/MainProcess] Received task: orders.tasks.order_created[a1d79ddb-eb2f-4791-9e8a-f1a71bf5f44e]
[2020-06-23 03:37:16,924: INFO/MainProcess] Received task: orders.tasks.order_created[afb84e39-3d10-42d0-ab9e-333917ff66f9]
[2020-06-23 03:37:16,927: INFO/MainProcess] Received task: orders.tasks.order_created[86dc7ad7-66e6-4f3a-9db1-6f2a5cb5791f]
[2020-06-23 03:37:16,931: INFO/MainProcess] Received task: myshop.celery.debug_task[8169ff3e-ceae-478f-87ab-4020bedb8338]
[2020-06-23 03:37:16,935: INFO/MainProcess] Received task: orders.tasks.order_created[e689b21e-78b6-498b-adb4-f64dedbbbac8]
[2020-06-23 03:37:16,938: INFO/MainProcess] Received task: orders.tasks.order_created[92db7aa9-bf16-4be1-9769-a1986ffc6f6d]
[2020-06-23 03:37:16,940: INFO/MainProcess] Received task: orders.tasks.order_created[5c0e9873-2671-49a6-9c53-db6dfdd041ef]
[2020-06-23 03:37:16,943: INFO/MainProcess] Received task: orders.tasks.order_created[8d17faa5-19d7-49ed-8181-5a334423c1bb]
[2020-06-23 03:37:18,420: INFO/MainProcess] Received task: orders.tasks.order_created[4671d8c9-11dd-4c9a-a7b1-d0cfedab5e0a]
[2020-06-23 03:37:18,443: INFO/MainProcess] Received task: orders.tasks.order_created[c8f588c4-6e21-4fc8-b441-b65456207cd6]
[2020-06-23 03:37:18,453: INFO/MainProcess] Received task: orders.tasks.order_created[509fd1ad-eac9-4aab-b1f1-1c6278ebbffc]
[2020-06-23 03:37:19,080: INFO/SpawnPoolWorker-9] child process 10652 calling self.run()
[2020-06-23 03:37:19,095: INFO/SpawnPoolWorker-12] child process 13548 calling self.run()
[2020-06-23 03:37:19,097: INFO/SpawnPoolWorker-11] child process 252 calling self.run()
[2020-06-23 03:37:19,112: INFO/SpawnPoolWorker-13] child process 4604 calling self.run()
[2020-06-23 03:37:19,119: INFO/SpawnPoolWorker-14] child process 12696 calling self.run()
[2020-06-23 03:37:19,121: INFO/SpawnPoolWorker-16] child process 13080 calling self.run()
[2020-06-23 03:37:19,125: INFO/SpawnPoolWorker-10] child process 12420 calling self.run()
[2020-06-23 03:37:19,159: INFO/SpawnPoolWorker-15] child process 4468 calling self.run()
[2020-06-23 03:37:21,177: INFO/SpawnPoolWorker-20] child process 14032 calling self.run()
[2020-06-23 03:37:21,183: INFO/SpawnPoolWorker-19] child process 11220 calling self.run()
[2020-06-23 03:37:21,183: INFO/SpawnPoolWorker-18] child process 1084 calling self.run()
[2020-06-23 03:37:21,191: INFO/SpawnPoolWorker-17] child process 14020 calling self.run()

and log after create a new task:

[2020-06-23 03:41:32,735: INFO/MainProcess] Received task: orders.tasks.order_created[758b7636-314b-40df-8458-d03bb1ca2ac2]
[2020-06-23 03:41:33,536: INFO/SpawnPoolWorker-22] child process 16052 calling self.run()
[2020-06-23 03:41:33,537: INFO/SpawnPoolWorker-21] child process 9664 calling self.run()
[2020-06-23 03:41:33,538: INFO/SpawnPoolWorker-24] child process 18172 calling self.run()
[2020-06-23 03:41:33,546: INFO/SpawnPoolWorker-26] child process 9704 calling self.run()
[2020-06-23 03:41:33,547: INFO/SpawnPoolWorker-23] child process 10612 calling self.run()
[2020-06-23 03:41:33,547: INFO/SpawnPoolWorker-25] child process 11176 calling self.run()
Tags:

3

Answers


  1. 0

    Given the logs I assume you are on Windows. However, since Celery 4.x Windows is no longer supported. Problems you are encountering are also described in this Celery issue and this SO question. The general discussion about running Celery 4.x on Windows can be found here. The solution to your problem seems to be to use e.g. gevent execution pool (more on execution pools can be found here). When using Django:

    python manage.py celery worker -P gevent
    Login or Signup to reply.
  2. 0

    In hope that this will help someone I also faced this problem, I use Django in its most recent version as well as Celery.
    In fact since version 4 of celery Windows is not supported to get around this problem you need another library like Gevent more information on this page.

    Login or Signup to reply.
  3. 0

    Windows does not support celery 4.0+,
    So To Solve this issue :-

    1. We have to install gevent using pip :-

      pip install gevent

    2. Then to run celery,

      celery -A <proj_name> worker -l info -P gevent

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search