skip to Main Content

I have created a project github repository that has Docker, Django, React, Redis, Celery:
https://github.com/axilaris/dockerize-django-react-celery-redis. My goal is to get celery working correctly with the logo appearing and background task log prints.

This is based from this Docker – React and Django example tutorial code:
https://github.com/dotja/authentication_app_react_django_rest

And trying to use Docker – Celery & Redis from this tutorial code:
https://github.com/veryacademy/docker-mastery-with-django/tree/main/Part-4%20Django%20Postgres%20Redis%20and%20Celery <– Part 4 tutorial for Celery & Redis

Here is the docker-compose.yaml for the redis & celery part:

# Redis
  redis:
    image: redis:alpine
    container_name: redis
    ports:
      - "6379:6379"

# celery
  celery:
    restart: always
    build:
      context: ./backend
    command: celery -A backend worker -l DEBUG
    volumes:
      - .:/django
    container_name: celery  
    depends_on:
      - redis
      - backend

here is how I implemented the background tasks in user_api/tasks.py (https://github.com/axilaris/dockerize-django-react-celery-redis/blob/main/backend/user_api/tasks.py)

in backend/user_api/tasks.py

from __future__ import absolute_import, unicode_literals

from celery import shared_task
import logging

@shared_task
def add(x, y):
    logging.debug("XXX add")
    return x + y

my project should work very simply by running:

 docker-compose build
 docker-compose up

However, there is no celery logo (1) and I dont see the logs print on the background process (2). I think it is executing as result.ready returns. But I want these 2 for completeness. (eventhough result.ready maybe be working with celery executing background task)

It did not print this celery logo. (you can check the full log prints for docker-compose up – https://gist.github.com/axilaris/a2776fc8f39e53bbc93e0d7a4e0ba0f5):

celery       |  -------------- celery@b755a7cdba8d v5.3.6 (emerald-rush)
celery       | --- ***** ----- 
celery       | -- ******* ---- Linux-6.6.12-linuxkit-aarch64-with 2024-03-02 20:48:06
celery       | - *** --- * --- 
celery       | - ** ---------- [config]
celery       | - ** ---------- .> app:         core:0xffff9bbd7550
celery       | - ** ---------- .> transport:   redis://redis:6379//
celery       | - ** ---------- .> results:     redis://redis:6379/
celery       | - *** --- * --- .> concurrency: 10 (prefork)
celery       | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
celery       | --- ***** ----- 
celery       |  -------------- [queues]
celery       |                 .> celery           exchange=celery(direct) key=celery

and here is how I execute the background task, but there is no log prints in django logs on the background task. (you can find the full log prints https://gist.github.com/axilaris/a2776fc8f39e53bbc93e0d7a4e0ba0f5)

>>> % docker exec -it backend_container sh 
>>> /app # python manage.py shell
>>> Python 3.9.18 (main, Jan 27 2024, 07:18:02) 
>>> [GCC 13.2.1 20231014] on linux
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> (InteractiveConsole)
>>> from user_api.tasks import add
>>> result = add.delay(2, 2)
>>> result.ready
<bound method AsyncResult.ready of <AsyncResult: 9046dd90-f44d-4eba-9881-acc0fbc4278a>>

My goal is to see that celery logo print above and to verify in the django logs that the background process is executed just like in (https://youtu.be/oBQxFn1CDno?si=58ZRfLZeuCC8fz01&t=1204 at 20:04).

2

Answers


  1. Not sure if you’re looking for logo such Redis ASCII logo shown below in your terminal.

                    _._                
               _.-``__ ''-._           
          _.-``    `.  `_.  ''-._      
      .-`` .-```.  ```/    _.,_ ''-._ 
     (    '      ,       .-`  | `,    )
     |`-._`-...-` __...-.``-._|'` _.-'|
     |    `-._   `._    /     _.-'    |
      `-._    `-._  `-./  _.-'    _.-' 
     |`-._`-._    `-.__.-'    _.-'_.-'|
     |    `-._`-._        _.-'_.-'    |
      `-._    `-._`-.__.-'_.-'    _.-' 
     |`-._`-._    `-.__.-'    _.-'_.-'|
     |    `-._`-._        _.-'_.-'    |
      `-._    `-._`-.__.-'_.-'    _.-' 
          `-._    `-.__.-'    _.-'     
              `-._        _.-'         
                  `-.__.-'             
    
    

    In any case, here’s what I went through.

    The full logs in your GitHub Gist shows that Celery is functioning correctly, but it doesn’t display the Celery logo because it’s a background process.

    Celery Functionality:

    • celery container creation is successful
    • celery starts the gunicorn worker ([2024-03-13 12:27:32 +0000] [10] [INFO] Starting gunicorn 20.1.0).
    • There are no errors related to Celery in the logs.

    Missing Celery Logo:

    Celery itself doesn’t have a graphical user interface (GUI) that displays a logo. It operates in the background, handling tasks asynchronously.

    Background Task Logs:

    The logs show successful execution of background tasks is likely because result.ready returns.

    Debug Logs:

    I see you enabled debugging mode in your docker-compose.yaml file and recommend to check the celery logs docker-compose logs celery --tail=1000 -f in order to check any missing logs.

    Please let me know what you expect to see such as svg logo or ASCII logs if something I’m missing.

    Login or Signup to reply.
  2. I noticed that on some machines i could only start celery if i provide also name of worker, scheduller, queue and concurrency, try to run celery as follows:

    celery worker -A backend --loglevel=info --concurrency=4 -n backend@localhost -Q backend -E -Ofair -B -s backend_schedule
    

    also don’t forget to configure celery at start like this:

    django.setup()
    broker = CELERY_BROKER_URL
    backend = CELERY_RESULT_BACKEND
    app = Celery('backend')
    app.config_from_object('django.conf:settings', namespace='CELERY')
    app.conf.broker_url = broker
    app.conf.result_backend = backend
    app.autodiscover_tasks()
    
    

    probably CELERY_BROKER_URL && CELERY_RESULT_BACKEND you should set from env variables, in my case i set them in settings file.

    Also it might be helpful to attach you image by id and try to start celery from cli.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search